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FOREWORD 


This  report  revises  and  supersedes  NAVSWC  TR  84-373,  Revision  2,  dated  March  1991  (which 
should  be  discarded).  The  revision  reflects  the  current  Statistical  Modeling  and  Estimation  of  Reliability 
Functions  for  Software  (SMERFS)  software  package.  The  program  has  undergone  many  enhancements 
and  modifications,  including  (1)  the  implementation  of  the  new  model  applicability  analyses,  such  as 
accuracy,  bias,  trend,  and  noise;  (2)  the  addition  of  the  Jelinski/Moranda  De-Eutrophication  model  for 
execution  time  data;  (3)  the  update  to  the  Schneidewind  model  for  interval  data  to  be  more  consistent  with 
his  current  theories;  and  (4)  the  addition  of  a  Goodness -of-Fit  routine  for  the  execution  time  models. 

The  report  has  been  reviewed  by  Dr.  Richard  Lorey,  Head,  Advanced  Computation  Technology 
Group.  Comments  concerning  this  technical  report  should  be  directed  to  the  Commander,  NSWCDD, 
Attn:  BIO,  Dahlgren,  Virginia  22448-5000. 


Approved  by: 

D.  B.  COLBY,  Head 

Systems  Research  and  Technology  Department 
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ABSTRACT 


This  is  the  third  in  a  series  of  Naval  Surface  Warfare  Center  Dahlgren  Division  (NSWCDD) 
technical  reports  concerning  software  reliability.  The  first  report,  A  Survey  of  Software  Reliability 
Modeling  and  Estimation.  NSWC  TR  82-171,  discusses  various  approaches  advocated  for  reliability 
estimation;  reviews  various  models  proposed  for  this  estimation  process;  provides  model  assumptions, 
estimates  of  reliability,  and  die  precision  of  those  estimates;  and  provides  the  data  required  for  die 
models’  implementation.  Eight  software  reliability  models  were  selected  to  form  die  basis  of  a  library. 
This  library  also  contains  data  edit,  transformation,  general  statistics,  and  Goodness-of-Fh  functions.  The 
original  Statistical  Modeling  and  Estimation  of  Reliability  Functions  for  Software  (SMERFS)  Library  was 
described  in  the  SMERFS  Library  Access  Guide.  NSWC  TR  84-371.  The  enhanced  library,  which  now 
contains  11  models  and  model  applicability  analyses,  is  explained  in  NSWCDD  TR  84-371,  Revision  3. 
The  execution  of  this  more  powerful  library,  through  die  new  SMERFS  driver,  is  explained  herein. 
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CHAPTER  1 

DOCUMENT  OVERVIEW 


1.1  INTRODUCTION 

This  report  is  designed  to  provide  supplemental  information  concerning  the  execution  of  the  revised 
Statistical  Modeling  and  Estimation  of  Reliability  Functions  for  Software  (SMERFS)  software  package. 
The  term  “supplemental"  is  used  because  SMERFS  actually  contains  its  own  user’s  guide.  SMERFS  is 
interactive  in  execution,  and  its  processing  is  controlled  through  the  user  selections  of  various  menu 
options  and  user  responses  to  program-generated  output  prompts.  Within  certain  areas,  the  user  may 
even  choose  to  let  die  program  list  the  assumptions  and  requirements  before  initiating  a  particular  portion 
of  the  program. 

Because  of  these  SMERFS  design  features,  this  report  is  not  really  a  user’s  guide  in  its  purest 
sense.  Its  primary  purpose  is  to  convey  die  processing  features  of  SMERFS  and  to  explain  (in  more 
detail)  die  generated  prompts  and  resulting  computational  outputs.  The  contents  of  this  report  may  be 
used  to  develop  a  master  testing  plan,  including  the  determination  and  collection  of  all  input  requirements 
for  the  SMERFS  program. 


1.2  HISTORICAL  BACKGROUND 

The  original  SMERFS  software  package  was  released  in  1984  and  consisted  of  the  SMERFS 
Library  (SMFLIB)  (Reference  1)  and  three  machine-dependent  drivers.  The  SMFLIB  contained  the 
statistical  and  mathematical  processing  required  to  obtain  the  various  software  reliability  estimates 
(Reference  2).  The  three  drivers  allowed  this  library  to  be  executed  in  an  interactive  mode  on  die  Naval 
Surface  Warfare  Center  Dahlgren  Division  (NSWCDD)  Control  Data  Corporation  (CDC)  6700,  die  CDC 
CYBER  170/760,  and  the  VAX  11/780  computer  systems.  For  other  systems,  die  authors  suggested 
creating  a  new  driver  or  modifying  the  driver  closest  to  the  system. 

In  1987,  die  first  revision  to  the  SMERFS  software  package  was  released.  That  revision  primarily 
affected  the  driver  and  its  documentation,  but  changes  were  also  made  to  the  library.  Perhaps  die  most 
important  modification  to  the  library  involved  the  discovery  of  an  error  in  die  implementation  of  die 
equations  for  the  second  and  third  treatment  types  of  Schneidewind’s  Maximum  Likelihood  model  for 
interval  data  analysis.  That  error  was  corrected  in  the  release  and  cited  in  the  first  revision. 

The  driver  for  that  SMERFS  software  package  was  completely  machine  independent;  it  being  a 
subset  of  the  American  National  Standards  Institute  (ANSI)  specifications  for  die  FORTRAN  77  compiler. 
To  transport  the  program  from  a  mainframe  (e.g.,  the  CDC  170/875  at  NSWCDD)  to  either  a 
minicomputer  (e.g.,  the  VAX  11/785)  or  any  IBM-compatible  Personal  Computer  (PC)  only  required 
changing  die  single  precision  declarations  to  double  precision.  The  implicit  variable  declarations  had  to 
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be  modified  to  double  precision  when  transporting  to  'smaller*  computer  systems  to  yield  the  same 
'good*  predictions  as  could  be  obtained  on  'larger*  computer  systems.  The  modification  to  the  variable 
declaration  line  had  to  be  performed  in  all  routines  of  the  driver  and  the  library. 

In  addition  to  the  feature  of  complete  machine  independence,  the  driver  was  modified  in  many 
other  ways.  It  included  two  additional  options.  An  internal  line  printer  plotter  had  been  added  for  users 
without  access  to  the  TEKTRONIX  PLOT- 10  Library  or  a  graphics  terminal.  Although  the  graphs  are 
crude,  they  can  still  assist  the  user  in  detecting  'trends*  in  the  data  or  residuals,  and  in  determining  how 
well  the  fitted  model  predicts  the  data. 

The  second  option  was  the  creation  of  a  file  of  all  program  prompts,  user  responses,  and  program 
output  over  a  single  SMERFS  execution.  Upon  completion  of  the  SMERFS  program,  the  file  can  be 
cataloged  for  future  documentation  purposes.  Also,  die  file  could  be  used  in  determining  user  input 
errors  during  execution. 

Substantial  efforts  were  also  made  to  reduce  the  amount  of  generated  program  prompts  (and 
therefore  user  responses).  Several  global  flags  were  added  to  the  driver  to  transfer  the  data  type,  the 
prediction  method,  and  other  information.  That  information  was  continually  requested  in  the  previous 
SMERFS  release.  These  flags  were  also  used  to  reduce  the  possibility  of  the  user  performing  an  analysis 
using  the  wrong  data. 

The  generated  output  of  the  SMERFS  software  package  had  also  undergone  some  cosmetic  cleanup, 
including  right-hand  justification  of  the  text,  and  (more  importantly)  the  addition  of  lead  asterisks  on  all 
error  messages. 

In  1990,  the  second  revision  to  the  SMERFS  software  package  was  released.  That  revision 
primarily  affected  the  library  and  its  documentation,  but  interface  changes  (caused  by  the  new  library) 
and  general  enhancements  were  also  made  to  the  driver. 

Prior  to  that  release,  the  software  reliability  models  included  in  the  SMFLIB  used  either  the  Nelder- 
Mead  algorithm  or  the  Newton-Raphson  procedure.  Those  numerical  methods,  in  addition  to  requiring 
initial  starting  values,  have  the  inherent  problem  of  "divergence.”  This  is  the  condition  in  which,  while 
attempting  to  converge  on  the  optimum  value  of  the  function  (if  it  exists),  the  method  actually  begins  to 
move  away  from  that  maximum  (minimum).  The  models  could  also  encounter  a  possible  division  by 
zero,  exponential  overflow,  or  exponential  underflow  while  attempting  to  converge  on  that  optimum 
value. 


The  prior  problems  were  almost  completely  eliminated  by  the  use  of  the  Dekker-Brent  and  Trust 
Region  algorithms  in  the  second  revision.  These  optimization  techniques  establish  most  of  the  initial 
starting  values,  are  more  able  to  handle  "misbehaving"  data,  and  converge  to  the  optimum  value  of  the 
function  more  often  and  more  readily. 

Two  additional  models  were  also  added  to  the  SMFLIB,  bringing  the  total  to  10.  Those  were  John 
Musa’s  Logarithmic  Poisson  Execution  model  (Reference  3)  and  Yamada’s  S-Shaped  Reliability  Growth 
model  (Reference  4,  pages  475-478). 

In  addition  to  the  modifications  required  because  of  the  new  SMFLIB,  the  driver  was  also  modified 
in  two  areas.  First,  the  three  routines  used  to  validate  the  model  fit  were  grouped  under  a  single 
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secondary  module.  Second,  an  optional  output  file  was  available  to  overcome  the  ’crudeness*  of  the 
SMERFS  graphic  utility,  while  retaining  the  achieved  complete  machine  independence.  The  file, 
containing  all  (selected)  data  used  in  plot  generations,  could  be  processed  (using  the  graphic  resources 
available  to  the  user)  to  produce  plots  of  high  quality. 

In  1993,  the  third  revision  to  the  SMERFS  software  package  was  released  (SMERFSS).  This 
revision  has  equally  affected  the  driver  and  the  library.  The  changes  to  the  SMFLIB  include  the  addition 
of  a  Goodness-of-Fit  analysis  for  execution  time  models  through  the  Kolmogorov-Smimov  Statistic,  the 
addition  of  the  Jelinski/Moranda  De-Eutrophication  model,  the  addition  of  several  model  applicability 
analyses  (Reference  S),  and  the  alteration  of  the  Schneidewind  model  to  be  more  consistent  with  his 
current  software  reliability  model  (Reference  6). 

The  changes  (specific)  to  the  driver  include  the  reduction  in  size  of  the  SMERFS  data  file  to  only 
the  number  of  observed  elements,  the  addition  of  the  units  for  Time-Between-Failure  (TBF)  data  types, 
the  input  of  an  American  Standard  Code  for  Information  Interchange  (ASCII)  file  of  data  (rather  than 
requiring  the  user  to  input  the  initial  data  via  the  keyboard  option),  and  the  additional  standardization 
of  model  estimations  and  predictions  (through  the  addition  of  many  equations  within  the  driver  using  the 
parameters  returned  from  the  SMFLIB).  Tables  1-1  and  1-2  list  the  values  generated  for  execution  time 
data  models  and  interval  data  models,  respectively.  The  abbreviations  in  the  column  headers  on  those 
tables  correspond  to  the  software  reliability  models,  as  follows: 

a.  GEO  -  Geometric  model 

b.  JAM  -  Jelinski/Moranda  De-Eutrophication  model 

c.  LAV  -  Littlewood  and  Verrall’s  Bayesian  Reliability  Growth  model 

d.  MUS  -  John  Musa’s  Basic  Execution  Time  model 

e.  MSA  -  John  Musa’s  Logarithmic  Poisson  model 

f.  NPT  -  Non-homogeneous  Poisson  model  (for  execution  time) 

g.  BAM  -  Brooks  and  Motley’s  Discrete  Software  Reliability  model 

h.  GPO  -  Generalized  Poisson  model 

i.  NPI  -  Non-homogeneous  Poisson  model  (for  interval  data) 

j.  SDW  -  Schneidewind’s  Maximum  Likelihood  model 

k.  ESH  -  Yamada’s  S-Shaped  Reliability  Growth  model 

Within  those  tables,  shaded  regions  indicate  new  or  enhanced  capabilities  over  the  previous  SMERFS 
revision,  checkmarks  indicate  values  printed  by  the  SMERFS  software  package,  and  stars  indicate  values 
calculated  in  the  driver  portion.  For  example,  the  Mean-Time-Before-Next-Failure  (MTBNF)  is  output 
for  the  first  five  Gisted)  models;  however,  the  values  for  the  third  and  fifth  are  computed  in  the  driver. 


1.3  OPERATIONAL  ENVIRONMENT 

The  SMERFS  program  is  currently  available  on  a  number  of  different  computers  at  NSWCDD, 
Dahlgren,  Virginia.  These  range  from  large  mainframes  to  PCs.  The  transfer  to  other  computers  should 
require  no  additional  modifications  or  alterations,  simply  recompilation  on  the  new  target  computer.  The 
program  is  constructed  using  only  a  subset  of  the  ANSI  specifications  for  the  FORTRAN  77  compiler. 
A  FORTRAN  77  compiler  (meeting  those  ANSI  specifications)  is  the  only  requirement  for  establishing 
the  program  on  another  computer  system,  in  addition  to  the  previously  discussed  single  and  double 
precision  declarations. 
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TABLE  1-1.  VALUES  GENERATED  FOR  EXECUTION  TIME  DATA  MODELS 


MODEL  ESTIMATES  AND 
PREDICTIONS 

GEO 

JAM 

LAV 

MUS 

MSA 

NPT 

ALPHA  Parameter 
(Model  Specific) 


BET  AO  Parameter 
(Model  Specific) 


BETA1  Parameter 
(Model  Specific) 


Proportionality  Constant 


Hazard  Rate  Parameter 


Initial  Intensity  Function 


Current  Intensity  Function 


Initial  MTBNF 


Current  MTBNF 


Total  Number  of  Faults 


Total  Number  of  Faults 
Remaining 


Purification  Level 


Expected  Reliability  for  a 
Specified  Time 


Time  to  Reach  a  Specified 
Reliability  for  a  Specified 
Operational  Time 


Total  MTBNF  for  the  Next  K 
Failures 


Time  and  Number  of  Failures 
to  Reach  a  Desired  Intensity 
Function 


Time  and  Number  of  Failures 
to  Reach  a  Desired  MTBNF 


Number  of  Failures  Expected 
in  a  Specified  Time 
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TABLE  1-2.  VALUES  GENERATED  FOR  INTERVAL  DATA  MODELS 


MODEL  ESTIMATES  AND 
PREDICTIONS 

BAM 

GPO 

NPI 

SDW 

ALPHA  Parameter 
(Model  Specific) 

✓ 

✓ 

BETA  Parameter 
(Model  Specific) 

✓ 

Proportionality  Constant 
(Model  Specific) 

✓ 

✓ 

✓ 

Probability  of  Detecting  Faults 

✓ 

Total  Number  of  Faults 

✓ 

✓ 

✓ 

✓ 

✓ 

Total  Number  of  Faults  Remaining 

✓* 

/ 

✓* 

Weighted  Least  Squares  Criteria 

✓ 

Mean  Square  Error  (Faults) 

Criteria 

✓ 

Mean  Square  Error  (Time)  Criteria 

✓ 

Number  of  Faults  Expected  in  the 
Next  K  Periods 

✓ 

✓ 

✓ 

✓ 

✓ 

Number  of  Periods  Needed  to 
Discover  the  Next  M  Faults 

✓* 

✓* 

✓* 

✓ 

Software  Reliability  in  Next  Period 

✓ 

As  for  the  establishment  of  the  program  on  a  PC,  the  authors  are  not  aware  of  any  IBM-compatible 
PC  on  which  the  program  executable  file  cannot  be  simply  loaded  and  executed.  SMERFS  does  not  make 
use  of  any  special  hardware  (e.g.,  a  math  coprocessor),  special  operating  system  features,  or  special 
peripheral  support  software  (e.g.,  plotters  or  database  managers). 


1.4  USER’S  GUIDE  ORGANIZATION 

This  report  is  organized  to  provide  easy  access  to  the  information  necessary  to  perform  a  software 
reliability  analysis  and  for  the  maintenance  of  the  files  of  failure  and  fault  data.  The  information  is 
divided  into  13  chapters.  A  user  should  be  able  to  easily  locate  and  examine  a  desired  section  and 
determine  what  additional  data  (if  any)  are  needed  and  the  potential  errors  that  may  occur. 

Chapter  2  contains  the  information  necessary  to  initiate  execution  of  the  program,  including  the 
necessary  computer  commands.  This  chapter  includes  the  first  menu  of  SMERFS,  which  conveys  the 
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nine  main  modules  (input,  edit,  unit  conversion,  transformation,  general  statistics,  raw  plot,  the  model 
applicability  analysis,  die  software  reliability  models,  and  the  analyses  of  model  fit). 

Chapters  3  through  1 1  contain  the  documentation  on  each  of  the  eight  main  modules.  The  contents 
of  the  eighth  main  module  (the  software  reliability  models)  are  divided  into  two  chapters  (hence  the  extra 
chapter).  There  are  actually  two  sets  of  models  within  SMERFS.  There  is  a  set  of  six  models  for 
execution  time  data  and  another  set  of  five  models  for  interval  data.  (Refer  to  Chapter  3  for  comments 
on  die  different  data  types,  and  Chapters  10  and  1 1  for  a  list  of  the  models.) 

Chapter  12  contains  the  analysis  of  model  fit  processing,  which  has  beat  changed  to  be  a  secondary 
module  of  die  eighth  module  (of  model  executions)  for  the  new  SMERFS5  program. 

Chapter  13  contains  the  SMERFS  termination  message  and  identifies  some  followup  processing  that 
may  be  required  for  various  computer  systems.  The  chapter  also  shows  the  format  and  describes  the  use 
of  the  optional  Plot  fde. 

Appendixes  A  and  B  show  actual  software  reliability  analyses  performed  on  a  PC  for  execution 
time  and  interval  data  types,  respectively.  (These  are  annotated  copies  of  the  optional  History  file.) 
Appendixes  C  and  D  contain  listings  of  the  optional  Plot  files  created  during  the  software  reliability 
analyses  contained  in  Appendixes  A  and  B. 


1.5  MODULE  DESCRIPTION  ORGANIZATION 

The  description  of  each  main  module  of  SMERFS  will  be  divided  into  two  or  three  sections.  The 
first  provides  an  introduction  to  the  module  and  its  potential  use.  (References  1  and  2  provide  additional 
useful  information  on  these  modules.) 

The  second  section  traces  the  input/output  0/0)  portion  of  the  SMERFS  execution.  This  section 
contains  listings  of  the  prompts  along  with  necessary  input  and  generated  output.  The  input  to  SMERFS 
(unless  otherwise  specified)  can  include  a  decimal  portion  (i.e.,  most  are  declared  to  be  real  variables  in 
SMERFS).  The  output  listings  of  SMERFS  are  not  limited  to  computational  results  only,  but  also  include 
the  various  SMERFS  menus,  the  error  messages  (both  informative  and  fatal  error  types),  and  the 
additional  listings  if  the  user  specifies  a  desire  for  the  expanded  assumptions  and  data  requirements. 

The  material  in  these  sections  is  presented  in  an  outline-style  format  containing  blocked  text.  This 
format  was  selected  to  allow  users  to  expediently  locate  particular  portions  of  the  SMERFS  execution 
output  from  the  guide. 

The  third  section  contains  descriptions  of  the  various  I/O  values.  It  is  not  applicable  to  all  chapters 
and  will  only  be  included  in  those  containing  the  descriptions  of  the  general  statistics,  software  reliability 
models,  and  analyses  of  model  fit  modules. 


1.6  FIGURE  DESCRIPTIONS 

To  facilitate  the  use  of  this  report,  several  figures  have  been  inserted  within  the  execution  flow 
section.  These  figures  will  be  placed  directly  after  the  text  explaining  the  content  of  the  figure.  The 
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figures  represent  the  cathode  ray  tube  screen  images  that  die  user  will  see,  but  they  are  not  die  result  of 
a  SMERFS  execution.  The  output  lines  have  been  modified  to  contain  only  the  fixed  portion  of  die  lines 
generated  by  SMERFS.  The  variable  portions,  containing  (during  execution)  computed  values  and 
Hollerith  identification  labels,  have  been  replaced  with  the  lowercase  letters  a,  e,  f,  and  i,  for  alphabetic, 
exponential,  floating  point,  and  integer  fields,  respectively.  These  place  holders  are  then  explained  in 
that  part  of  the  text  associated  with  that  figure. 

The  type  of  print  in  the  figures  conveys  additional  information  to  the  user  (Figure  1-1).  The  first 
print  style  in  the  figure  indicates  that  SMERFS  has  issued  a  prompt  and  that  a  response  must  be  entered. 
The  second  print  style  is  not  a  line  generated  by  SMERFS,  but  is  rather  an  inserted  line  indicating  that 
one  of  two  or  more  lines  will  appear.  This  strategy  was  adapted  to  lend  more  cohesiveness  to  the 
document  by  reducing  the  number  of  figures  in  the  text.  The  third  print  style  is  used  in  all  other  program 
output.  These  include  the  computed  values,  error  messages,  and  extended  descriptions. 


STYLE  1  - 

BOLD 

-  Program  Prompt 

STYLE  2  - 

SCRIPT 

-  Inaartad  Conditional  Haaaaga 

STYLE  3  - 

REGULAR 

-  Program  Output 

FIGURE  1-1.  PRINT  STYLES 


1.7  PROGRAM  INPUTS,  OUTPUTS,  AND  ERROR  DETECTION  FACILITIES 

The  SMERFS  program  does  not  provide  the  user  with  a  totally  fool-proof  environment  to  perform 
software  reliability  analysis;  however,  it  will  provide  the  user  with  an  excellent  environment  if  the 
following  items  are  understood  prior  to  activation. 

a.  Prompted  values  (unless  otherwise  specified)  are  read  in  free-format.  This  means  that  spacing 
does  not  matter  and  spaces  or  commas  should  be  used  to  separate  values  when  more  than  one 
value  is  prompted. 

u.  The  program  does  not  perform  rounding  of  the  computed  estimates  and  predictions;  however, 
certain  tests  are  conducted  as  if  rounding  were  performed.  For  example,  if  the  model  estimate 
for  the  total  number  of  faults  is  305.44  for  a  data  set  containing  305  faults,  then  future 
predictions  are  not  allowed  since  the  rounded  value  indicates  all  faults  have  been  detected. 
Conversely,  if  the  model  estimate  for  the  remaining  number  of  faults  is  0.61,  then  future 
predictions  are  allowed  since  the  rounded  value  indicates  a  fault  remains.  This  explains  the 
apparent  infeasibility  of  the  prompt: 

"ENTER  VALUE  OF  M  (BETWEEN  ONE  AND  .61 134E+00),  OR  ZERO  TO  END." 
as  one  is  the  only  valid  number  for  M. 
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c.  Prompted  values  are  not  "parsed'*  to  ensure  that  the  value  is  a  legal  character  for  the  specific 
variable.  For  example,  entry  of  a  letter  for  an  integer  value  will  result  in  a  FORTRAN  error 
and  the  program  will  terminate. 

d.  Prompted  values  are  (usually)  not  qualified  to  be  appropriate  when  the  range  is  specified  in  die 
prompt.  For  example,  entry  of  two  for  the  starting  estimate  of  the  probability  of  detecting 
errors  (zero  <  input  value  <  one)  will  not  be  detected  and  the  program  may  terminate. 

e.  The  entered  values  for  the  lengths  of  testing  intervals  should  be  altered  as  values  larger  than 
zero,  and  die  number  of  faults  detected  in  those  testing  periods  should  be  entered  as  whole 
numbers  (i.e.,  testing  lengths  of  zoo  and  fault  counts  with  fractional  portions  will  result  in 
FORTRAN  errors  and  die  program  will  terminate). 
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CHAPTER  2 

SMERFS  ACCESS  AND  ACTIVATION 


2.1  INTRODUCTION 

SMERFS  is  designed  to  provide  a  simple,  but  extensive,  software  reliability  analysis.  The  program 
has  been  written  for  interactive  executions  and  includes  both  prompts  and  complete  error  detection 
facilities  with  error  message  generation.  Perhaps  the  single  most  important  decision  dial  the  user  will 
have  to  make  is  foe  type  of  data  collection  to  be  implemented.  SMERFS  is  designed  to  operate  on  three 
types  of  error  data:  Wall  Clock  (WQ  TBF,  Central  Processing  Unit  (CPU)  TBF,  and  interval  fault  count 
and  testing  length  data.  The  user’s  decision  on  which  error  type  is  to  be  performed  may  be  aided  by 
consulting  Reference  2,  and  Chapters  3,  10,  and  11  of  this  report. 


2.2  EXECUTION  FLOW 

This  section  explains  foe  commands,  prompts,  and  inputs  used  to  initiate  foe  SMERFS  execution. 
(Refer  to  Chapter  13  for  foe  followup  instructions  needed  for  proper  data  file  maintenance.)  The 
placement  of  and  foe  print  in  foe  figures  conform  to  foe  strategy  introduced  in  Sections  1.5  and  1.6. 

a.  Commands  from  foe  Operating  System  Level 

As  with  all  programs,  foe  SMERFS  program  is  executed  from  foe  operating  system  level  of 
foe  target  computer.  Although  foe  commands  vary  across  computers,  one  basic  requirement 
is  that  foe  program  executable  file  and  any  input  data  files  have  to  be  on  a  media  accessible 
to  foe  user.  For  foe  CDC,  this  implies  that  all  are  local  to  foe  interactive  session.  For  foe 
VAX,  this  implies  that  all  are  within  foe  user’s  directory  or  within  a  "permitted*  directory; 
for  foe  PC,  this  implies  that  all  are  resident  on  foe  computer  (preferably  on  foe  hard  disk). 
Figure  2-1  shows  foe  activation  command  for  foe  VAX  and  PC. 


TOribbo^SftCODiti/TbSyAXCLOSTBR,  tho  program  i*  executed  byt 

fKinr  UDISK7 I  [ OSMXTH. SMERFS. RKV5 .SHIPMENT JSMKRFS5 

Tor  tho  IBH-compmtlblo  PCs,  sssiming  the  program  hss  boon 
losdod  on  tho  hard  disk  (C),  the  program  is  oxocutod  byt 

CtSMERPSS 


FIGURE  2-1.  PROGRAM  ACTIVATIONS 
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b.  Program  Banner 

Once  activated,  the  program  responds  with  the  SMERFS  banner  line,  including  die  software 
revision  number  and  the  release  date  (Figure  2-2). 
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SOFTWARE  REVISION  NUMBER 

FIVE  (21  SEPTEMBER  1993) 

FIGURE  2-2.  PROGRAM  BANNER 


c.  History  File  Prompt 

The  program  prompts  for  the  file  name  of  the  History  file,  the  number  zero  if  die  file  is  not 
wanted,  or  die  number  one  for  details  on  the  file  (Figure  2-3).  As  shown,  this  file  contains 
the  program  prompts,  user  responses,  and  program  outputs.  (Refer  to  Appendixes  A  and 
B  for  examples  of  this  file  and  to  Chapter  13  for  followup  commands.) 

ERROR:  If  the  input  file  name  is  in  error  (as  determined  by  die  target  computer’s 

operating  system),  the  error  message  shown  at  the  bottom  of  the  figure  is  issued 
and  die  prompt  is  reissued. 


urn  OUTPUT  PILE  MANX  FOR  THE  HISTORY  PILE;  EERO  IF  THE  FILE  IS 
■OT  DESIRED,  OR  ONE  FOR  DETAILS  ON  THE  FILE. 

THE  HISTORY  FILE  IS  A  COPY  OF  THE  ENTIRE  INTERACTIVE  SESSION.  IT 
CAN  BE  USED  FOR  LATER  ANALYSIS  AND/OR  DOCUMENTATION. 

**FILE  MAKE  ERROR;  TRY  AGAIN  (AFTER  THE  PROMPT). 


FIGURE  2-3.  HISTORY  FILE  PROMPT 


d.  Plot  File  Prompt 

The  program  prompts  for  die  file  name  of  die  Plot  file,  the  number  zero  if  the  file  is  not 
warned,  or  the  number  one  for  details  on  the  file  (Figure  2-4).  As  shown,  this  file  contains 
data  values  to  be  processed  in  a  subsequent  execution  to  produce  high-quality  graphs  using 
various  plotting  libraries  and  graphic  devices  available  to  the  user.  (Refer  to  Chapter  12, 
step  g  for  instructions  for  placing  data  on  this  file,  and  Chapter  13  for  possible  followup 
commands,  file  format,  and  processing  instructions.) 
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NOTE:  It  is  emphasized  that  this  file  merely  contains  sequential  data  values  (as  shown 

in  Appendixes  C  and  D).  It  does  not  contain  actual  graphic  images. 

ERROR:  If  the  input  file  name  is  in  error  (as  determined  by  the  target  computer’s 

operating  system),  the  error  message  shown  at  the  bottom  of  the  figure  is  issued 
and  the  prompt  is  reissued. 


IBM  OUTPUT  FILE  NAME  FOR  THE  PLOT  FILE;  EBRO  IF  THE  FXXJt  IS 
HOT  DESIRED,  OR  (WE  FOR  DETAILS  ON  THE  FILE. 

THE  PLOT  FILS  CONTAINS  SELECTED  DATA  AND  LABELS  TO  ALLOW  A  USER- 
SUPPLIED  GRAPHICS  PROGRAM  TO  GENERATE  HIGH-QUALITY  PLOTS.  SINCE 
A  CHARACTER  PLOTTER  IS  IMPLEMENTED  WITHIN  THE  SMERFS  PROGRAM  (TO 
ENSURE  MACHINE  PORTABILITY  OF  THE  PACKAGE ) ,  THE  USE  OF  THIS  OP¬ 
TION  IS  HIGHLY  RECOMMENDED. 

••FILE  NAME  ERROR;  TRY  AGAIN  (AFTER  THE  PROMPT). 


FIGURE  2-4.  PLOT  FILE  PROMPT 


e.  Data  Type  Selection 

The  program  then  prompts  the  user  to  enter  the  type  of  data  to  be  processed  (Figure  2-5). 
If  an  invalid  number  is  entered,  the  same  list  and  second  prompt  are  output.  (Refer  to 
Chapter  3  for  comments  on  the  different  data  types,  and  Chapters  10  and  1 1  for  information 
on  the  models  available  for  the  different  data  types.) 


ENTER  DESIRED  DATA  TYPE,  OR  EERO  FOR  A  LIST. 
THE  AVAILABLE  DATA  TYPES  ARE: 

1  WALL  CLOCK  (WC)  TIME-BETWEEN-FAILURES  (TBF) 

2  CENTRAL  PROCESSING  UNITS  (CPU)  TBF 

3  WC  TBF  AND  CPU  TBF 

4  INTERVAL  FAULT  COUNTS  AND  TESTING  LENGTHS 

ENTER  DESIRED  DATA  TYPE. 


FIGURE  2-5.  DATA  TYPE  PROMPTS 


f.  Standard  SMERFS  File  Input 

The  program  then  prompts  for  a  flag  indicating  if  the  input  of  a  standard  SMERFS  data  file 
is  desired  (Figure  2-6).  If  a  data  file  from  a  previous  SMERFS  execution  is  not  available, 
enter  zero.  Otherwise,  if  a  data  file  is  to  be  input,  then  an  affirmative  response  should  be 
made  and  the  program  issues  the  second  prompt  for  the  WC  TBF,  CPU  TBF,  or 
INTERVAL  file  name.  The  program  then  responds  with  the  number  of  elements  read  and 
the  units  of  the  TBF  measurement.  If  the  data  type  was  entered  as  a  three,  processing  for 
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both  TBF  data  files  occurs.  The  program  then  automatically  transfers  control  to  the  Input 
module  (Chapter  3)  independent  of  the  availability  of  a  standard  data  file. 

NOTE:  It  was  not  until  the  fifth  revision  of  the  SMERFS  program  that  this  standard 

SMERFS  input  file  was  altered  to  contain  the  measurement  unit  for  die  TBF 
data;  entry  of  a  file  created  by  a  previous  revision  results  in  the  additional 
prompt  for  the  units  of  measurement. 


ENTER  OMB  FOR  A  STANDARD  SMERFS  FILM  INPUT;  ELSE  SERO. 

If  an  affirmative  raaponaa  waa  entered,  t ham 
ENTER  INPUT  PILE  NANS  FOR  an aaaaaa  DATA. 

**FILE  NAME  ERROR;  TRY  AGAIN  (AFTER  THE  PROMPT). 

If  antaring  a  TBF  film  created  by  a  previoua  SMERFS  raviaion ,  thaw 

THE  AVAILABLE  UNITS  FOR  TBF  DATA  PROCESSING  ARE: 

1  SECONDS  3  HOURS  5  WEEKS  7  YEARS 

2  MINUTES  4  DAYS  6  MONTHS 

ENTER  CODE  NUMBER  FOR  THE  aaaaaaaa  DATA. 

THE  INPUT  OF  ill  aaaaaaaa  ELEMENTS  WAS  PERFORMED. 

Elaa,  if  antaring  a  TBF  file  craatad  by  the  SMERFS5  r avia ion,  t ham 

THE  INPUT  OF  ill  aaaaaaaa  ELEMENTS  WAS  PERFORMED. 

THESE  TBF  DATA  WERE  STORED  IN  aaaaaaa 

Elaa,  If  antaring  an  interval  file,  than s 

THE  INPUT  OF  ill  aaaaaaaa  ELEMENTS  WAS  PERFORMED. 

End  if 

End  if 


FIGURE  2-6.  STANDARD  SMERFS  FILE  INPUT  PROMPTS 


g.  Data  File  Creations 

The  program  then  prompts  for  a  flag  indicating  if  the  creation  of  the  data  file(s)  is  desired 
(Figure  2-7).  Note,  this  prompt  only  occurs  once  in  the  program  execution;  a  negative 
response  means  that  the  data  file(s)  cannot  be  generated  at  a  later  point  within  the  execution. 
If  die  intent  is  to  eventually  save  the  data,  the  file  must  be  created  now  (even  if  an  error  was 
made).  If  the  file  creation  is  desired,  the  program  issues  the  second  prompt  for  the  WC 
TBF,  CPU  TBF,  or  INTERVAL  file  name  to  be  assigned.  The  program  then  responds  with 
the  number  of  elements  stored  and  the  units  of  the  TBF  measurement.  If  die  data  type  was 
entered  as  a  three,  processing  for  both  TBF  data  files  occurs.  Upon  completion,  the 
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program  indicates  the  number  of  elements  written  to  the  file  with  the  message  shown  in  the 
next-to-last  lines  of  the  figure.  (Refer  to  Chapter  13  for  followup  commands.) 


ERROR:  If  the  input  file  name  is  in  error  (as  determined  by  the  target  computer’s 

operating  system),  the  error  message  shown  at  the  bottom  of  the  figure  is  issued 
and  the  prompt  is  reissued. 


ENTER  OHS  FOR  THB  PROGRAM  TO  MAKE  NSW  DATA  FILES;  ELSE  SSR0.  TEE 
RESPONSE  WILL  BE  USED  THROUGHOUT  THB  EXECUTION.  A  SEED  WILL  ALSO 
VOID  THB  DATA  RESTORE  OPTION  IN  DATA  TRANSFORMATIONS. 

ENTER  OUTPUT  FILE  KANE  FOR  a*******  DATA. 

THE  FILE  STORAGE  OF  ill  laium  ELEMENTS  WAS  PERFORMED. 

THESE  TBF  DATA  ARE  STORED  IN  ******* 

••FILE  NAME  ERROR;  TRY  AGAIN  (AFTER  THE  PROMPT) . 


FIGURE  2-7.  FILE  CREATION  OPTION  AND  OUTPUT  MESSAGES 


h.  Main  Module  Menu 


The  program  prompts  the  user  to  enter  the  Main  module  option  number  (Figure  2-8).  If  an 
invalid  option  number  is  entered,  the  same  list  and  the  second  prompt  are  output. 


ENTER  MAIM  MODULE  OPTION,  OR  ZERO  FOR  A  LIST. 

THE  AVAILABLE  MAIN  MODULE  OPTIONS  ARE: 

1  DATA  INPUT  6  PLOT(S)  OF  THE  RAW  DATA 

2  DATA  EDIT  7  MODEL  APPLICABILITY  ANALYSES 

3  UNIT  CONVERSIONS  8  EXECUTIONS  OF  THE  MODELS 

4  DATA  TRANSFORMATIONS  9  STOP  EXECUTION  OF  SMERFS 

5  DATA  STATISTICS 
ENTER  MAIN  MODULE  OPTION. 


FIGURE  2-8.  MAIN  MODULE  MENU 


i.  Transfer  to  the  Indicated  Module 

The  program  then  transfers  control  to  the  indicated  module,  where: 

1.  The  Input  module  is  contained  in  Chapter  3. 

2.  The  Edit  module  is  contained  in  Chapter  4. 

3.  The  Unit  Conversions  module  is  contained  in  Chapter  5. 

4.  The  Transformations  module  is  contained  in  Chapter  6. 

5.  The  General  Statistics  module  is  contained  in  Chapter  7. 
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6.  The  Raw  Data  Plot  module  is  contained  in  Chapter  8. 

7.  The  Model  Applicability  module  is  contained  in  Chapter  9. 

8.  The  Model  Execution  module  for  execution  time  data  is  contained  in  Chapter  10  and 
the  module  for  interval  data  is  contained  in  Chapter  11. 

9.  The  Stop  Execution  module  is  contained  in  Chapter  13. 
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CHAPTER  3 
INPUT  MODULE 


3.1  INTRODUCTION 

The  Input  module  provides  for  the  input  of  data  via  both  a  previously  created  ASCII  file  or  the 
computer  keyboard.  Under  either  method  of  input,  the  data  are  appended  to  those  currently  existing  in 
the  program,  [i.e.,  if  a  "standard"  SMERFS  data  file  was  entered  at  the  program  activation  (Chapter  2, 
step  f),  the  values  entered  at  this  point  in  die  execution  will  be  appended  to  those.  If  two  ASCII  files  are 
entered,  then  SMERFS  operates  on  the  composition  of  die  two  (after  die  second  input).]  SMERFS  does 
not  .contain  a  re-start  capability  to  change  the  data  type  or  bring  m  a  completely  new  dam  set.  The 
program  must  be  ended  before  a  different  data  set  can  be  analyzed. 

Users  may  find  this  module  useful  in  performing  a  segmented  analysis  over  large  error  data  sets. 
[For  example,  if  die  error  collection  contains  100  intervals  of  testing,  75  may  be  entered  (from  an  ASCII 
file)  and  a  complete  reliability  analysis  performed.  Die  remaining  25  intervals  may  then  be  entered  (from 
another  file  or  by  the  keyboard)  and  die  reliability  analysis  re-conducted.)  In  this  manna:,  the  "growth" 
of  the  software  reliability  may  be  established.  The  module  will  also  be  useful  for  maintaining  die  file 
of  error  data  over  the  testing.  Here,  the  standard  SMERFS  file  should  be  ottered  in  Chapter  2,  step  f, 
and  the  current  updates  would  be  made  through  the  input  of  an  ASCII  file  or  die  keyboard  portion  of  the 
program. 


3.2  EXECUTION  FLOW 

This  section  explains  the  prompts  and  inputs  for  the  input  portion  of  SMERFS.  The  placement  of 
and  the  print  in  die  figures  conform  to  the  strategy  introduced  in  Sections  1.5  and  1.6. 

a.  Input  Option  Selection 

The  program  prompts  the  user  to  enter  the  type  of  data  input  to  be  performed  (Figure  3-1). 
If  an  invalid  option  number  is  entered,  the  same  list  and  second  prompt  are  output.  Briefly, 
the  ASCII  file  input  option  implies  that  the  user  has  entered  die  error  history  of  either  the 
execution  times  expended  between  consecutive  failures  or  the  interval  data  of  fault  counts 
and  testing  lengths  via  a  program  other  than  the  SMERFS  program  (e.g.,  a  clipboard  editor), 
and  desires  to  input  that  file  for  analysis  by  SMERFS.  SMERFS  accepts  these  through 
"free-format"  reads,  so  die  number  of  records  on  the  lines  and  the  number  of  lines  in  die 
file  do  not  affect  the  input.  Interval  data  has  the  extra  restriction  that  the  values  must  be 
entered  in  pairs  (where  the  first  value  reflects  the  fault  count  and  die  second  die  interval 
length). 
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FIGURE  3-1.  INPUT  MENU 


b.  Transfer  to  the  Indicated  Option 

If  the  entered  input  option  number  is  four,  control  returns  to  either  the  Data  File  Creations 
prompt  for  the  first  access  or  the  Main  Module  Menu  for  subsequent  accesses  (Chapter  2, 
steps  g  and  h,  respectively).  Otherwise  (if  a  valid  input  option  number  is  entered),  the 
program  transfers  control  to  the  indicated  input  option,  where: 

1.  The  ASCII  Input  option  is  contained  in  Paragraph  3.2.1. 

2.  The  Keyboard  Input  option  is  contained  in  Paragraph  3.2.2. 

3.  The  Data  Listing  option  is  contained  in  Paragraph  3.2.3. 


3.2.1  ASCII  File  Input 

If  the  entered  input  option  number  is  one,  then: 

a.  If  TBF  data  are  resident  in  the  program,  SMERFS  indicates  die  characteristics  of  the 
currently  stored  WC  TBF  and/or  CPU  TBF  data  set(s),  including  the  number  of  entries  and 
the  measurement  units  as  listed  under  step  c.  SMERFS  then  allows  the  user  to  abort  the 
input  if  the  current  file  does  not  agree  with  the  stored  data  (Figure  3-2).  If  the  response 
indicates  a  desire  to  abort  the  input,  control  returns  to  Section  3.2,  step  a. 


FIGURE  3-2.  ASCII  TIME  UNIT  CONSISTENCY  PROMPT 
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b.  The  program  prompts  for  the  WC  TBF,  CPU  TBF,  or  INTERVAL  file  name  of  the  ASCII 
file  (Figure  3-3).  If  the  data  type  was  entered  as  a  three,  processing  for  both  TBF  data  files 
occurs.  Users  are  reminded  that  the  use  of  the  ASCII  file  option  for  execution  time  data 
implies  that  the  file  contains  TBF  data;  the  module  is  not  designed  to  merge  testing  times 
until  a  failure  occurs,  as  is  performed  during  the  keyboard  input  (Paragraph  3.2.2).  For 
interval  data  files,  the  data  must  be  entered  in  pairs  (where  the  first  value  reflects  the  fault 
count  and  the  second  the  interval  length). 

ERROR:  If  the  input  file  name  is  in  error  (as  determined  by  the  target  computer’s 
operating  system),  the  error  message  shown  at  the  bottom  of  the  figure  is  issued 
and  the  prompt  is  reissued. 


ENTER  INPUT  PILE  NAME  FOR  ailMtu  DATA. 

•‘FILE  NAME  ERROR;  TRY  AGAIN  (AFTER  THE  PROMPT). 


FIGURE  3*3.  ASCII  FILE  NAME  PROMPT 


c.  If  this  is  the  first  entry  of  the  TBF  data,  SMERFS  prompts  for  the  code  number  of  the  unit 

measurement  (Figure  3-4).  If  the  data  type  was  entered  as  a  three,  processing  for  both  TBF 
data  files  occurs. 


FIGURE  3-4.  ASCII  TIME  UNIT  SPECIFICATION  PROMPT 


d.  The  program  then  responds  with  a  message  indicating  the  number  of  values  read  from  the 
specified  file  (Figure  3-5).  If  the  data  type  was  entered  as  a  three,  processing  for  both  TBF 
data  files  occurs. 

ERROR:  If  the  number  of  entries  exceeds  1000,  the  message  shown  at  the  bottom  of  the 
figure  is  issued  and  control  returns  to  Section  3.2,  step  a,  leaving  the  data 
vector(s)  unchanged. 
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THE  INPUT  OF  iii  aaaaaaaa  ELEMENTS  NAS  PERFORMED. 

••THE  NUMBER  OF  ENTRIES  EXCEEDS  THE  MAXIMUM  ARRAY  SIZE  FOR  SMERFS. 


FIGURE  3-5.  FILE  SIZE  MESSAGES 


e.  If  the  TBF  fatality  flag  indicates  that  the  final  time  entry  of  the  stored  data  did  not 
correspond  with  a  failure  occurrence,  the  first  entry  on  die  file  input  is  added  to  the  last 
stored  entry,  and  the  message  shown  in  Figure  3-6  is  issued. 


It  failure-tree  teat  time  exists  tor  the  lest  stored  entry ,  them 

THE  FIRST  VALUE  FROM  THE  FILE  WILL  BE  ADDED  TO  THE  LAST  ELEMENT 
OF  THE  STORED  DATA,  SINCE  THE  FATALITY  FLAG  INDICATES  A  FAILURE 
HAD  NOT  HAPPENED  ON  THE  FINAL  ENTRY. 

End  if 


FIGURE  3-6.  EXECUTION  TIME  ADDITION  MESSAGE 


f.  The  program  then  issues  a  message  indicating  the  total  number  of  values  resident  in  die 
storage  file(s)  after  the  new  values  are  added  (Figure  3-7).  If  the  data  type  was  entered  as 
a  three,  processing  for  both  TBF  data  files  occurs.  Additionally,  if  TBF  data  are  present, 
the  program  issues  a  prompt  to  determine  if  the  new  final  time  entry  corresponds  with  a 
software  failure  occurrence.  Upon  completion,  control  returns  to  Section  3.2,  step  a. 

ERROR:  If  the  total  number  of  entries  exceeds  1000,  the  message  shown  in  the  middle  of 
the  figure  is  issued  and  control  returns  to  Section  3.2,  step  a,  leaving  the  data 
vector(s)  unchanged. 


THE  aaaaaaaa  STORAGE  SPACE  NOW  CONTAINS  iii  ELEMENTS. 

••THE  ADDITION  WILL  EXCEED  THE  MAXIMUM  ARRAY  SIZE  FOR  SMERFS. 

It  TBF  data  are  resident  in  the  program,  then: 

ENTER  ONE  IF  A  FAILURE  HAPPENED  AT  THE  END  OF  THE  FINAL  ENTRY, 
OR  EBRO  IP  A  FAILURE  HAD  NOT  HAPPENED. 

End  it 


FIGURE  3-7.  TOTAL  STORAGE  MESSAGE  AND  FATALITY  FLAG  PROMPT 
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3.2.2  Keyboard  Input 

If  the  entered  input  option  number  is  two,  then: 

a.  If  TBF  data  are  resident  in  the  program,  SMERFS  indicates  the  characteristics  of  die 
currently  stored  WC  TBF  and/or  CPU  TBF  data  set,  including  the  number  of  entries  and 
the  measurement  units  as  listed  under  step  b.  SMERFS  then  allows  the  user  to  abort  the 
input  if  the  current  file  does  not  agree  with  the  stored  data  (Figure  3-8).  If  die  response 
indicates  a  desire  to  abort  the  input,  control  returns  to  Section  3.2,  step  a. 

NOTE:  If  WC  TBF  data  are  being  input,  SMERFS  issues  the  extra  message  that  the 
measurement  units  only  pertain  to  the  data  storage.  The  program  still  assumes 
that  the  WC  values  will  be  entered  in  24  hour  units.  This  is  explained  further 
in  step  c. 


If  TBF  da fca  ara  resident  In  the  program,  then: 

THE  FOLLOWING  DATA  ARE  CURRENTLY  RESIDENT. 

DATA  TYPE  NUMBER  UNITS 

aaa  TBF  ill  aaaaaaa 

ENTER  ONE  IF  THE  DATA  TO  BE  ENTERED  ARE  CONSISTENT  IN  TYPE  AND 
UNITS  WITH  THESE;  ELSE  ZERO  TO  ABORT  THIS  PROCESSING. 

If  WC  TBF  data  are  present ,  then: 

NOTE  WC  TBF  DATA  ARE  ENTERED  IN  24  HOUR  UNITS  AND  STORED  IN  THE 
UNITS  OF  THE  CURRENT  DATA,  aaaaaaa. 

End  if 

End  if 


FIGURE  3-8.  KEYBOARD  TIME  UNIT  CONSISTENCY  PROMPT 


b.  If  this  is  the  first  entry  of  the  TBF  data,  SMERFS  prompts  for  the  code  number  of  the  unit 

measurement  (Figure  3-9).  If  the  data  type  was  entered  as  a  three,  processing  for  both  TBF 
data  files  occurs. 

NOTE:  If  WC  TBF  data  are  being  input,  SMERFS  issues  the  extra  message  that  the 

measurement  units  only  pertain  to  the  data  storage.  The  program  still  assumes 
that  the  WC  values  will  be  entered  in  24  hour  units.  This  is  explained  further 
in  step  c. 
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If  TBF  data  vara  specified  and  data  arm  not  resident  in  tha  program,  t bant 

THE  AVAILABLE  UNITS  FOR  TBF  DATA  PROCESSING  AREt 
1  SECONDS  3  HOURS  S  WEEKS  7  YEARS 

3  MINUTES  4  DAYS  6  MONTHS 

ENTER  CODE  NUMBER  FOR  THE  aaaaaaaa  DATA. 

If  VC  TBF  data  warm  specified ,  then: 

NOTE  WC  TBF  DATA  ARE  ENTERED  IN  24  HOUR  UNITS  AND  STORED  IN  THE 
SPECIFIED  UNITS ,  aaaaaaa. 

End  if 

End  if 


FIGURE  3-9.  KEYBOARD  TIME  UNIT  SPECIFICATION  PROMPT 


c.  If  WC  TBF  data  sure  being  input  (data  type  was  entered  as  a  one),  SMERFS  issues  a  message 

telling  the  user  how  to  terminate  the  input  sequence  (Figure  3-10).  It  then  repeatedly 
prompts  for  the  starting  and  ending  times  of  testing  (24-hour  units),  die  number  of  failures 
observed  within  that  session,  and,  if  any  software  failures  were  observed,  die  time(s)  of  die 
failure(s)  (again,  in  24-hour  units).  SMERFS  then  converts  these  actual  times  into  the  TBF 
execution  units  as  directed  in  step  a  or  b.  Upon  completion  (as  indicated  through  either  user 
direction  or  error),  control  returns  to  Section  3.2,  step  a. 

ERROR:  The  current  SMERFS  software  can  handle  up  to  1000  entries,  which  may  equate 
to  thousands  of  testing  sessions;  the  program  continually  merges  the  testing  times 
until  a  failure  is  observed.  When  the  number  of  entries  exceeds  1000,  the 
message  shown  at  the  bottom  of  the  figure  is  issued  and  control  returns  to 
Section  3.2,  step  a. 


A  RESPONSE  OF  NEGATIVE  VALUES  FOR  THE  PROMPT* 

"ENTER  STARTING  &  ENDING  TIMES;  AND  NUMBER  OF  FAILURES." 

WILL  END  THE  PROCESSING. 

ENTER  STARTING  £  ENDING  TIMES;  AND  NUMBER  OF  FAILURES. 

BMTER  TIME  OF  FAILURE  NUMBER  ill. 

**THE  NUMBER  OF  ENTRIES  EXCEEDS  THE  MAXIMUM  ARRAY  SIZE  FOR  SMERFS. 


FIGURE  3-10.  WC  DATA  INPUT  PROMPTS 


d.  If  CPU  TBF  data  are  being  input  (data  type  was  entered  as  a  two),  SMERFS  issues  a 
message  telling  the  user  how  to  terminate  the  input  sequence  (Figure  3-11).  It  then 
repeatedly  prompts  for  the  expended  CPU  time  and  a  flag  of  zero  or  one.  No  unit 
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conversions  are  performed  on  the  input  values;  hence,  the  values  input  must  agree  with  those 
identified  in  step  a  or  b.  A  flag  of  one  indicates  that  the  testing  session  was  terminated  with 
a  failure  occurrence;  a  zero  indicates  that  a  failure  was  not  observed.  If  a  value  other  than 
zero  or  one  was  entered,  the  program  issues  the  message  in  the  next-to-last  line  of  the  figure 
and  the  flag  is  re-entered.  Upon  completion  (as  indicated  through  either  user  direction  or 
error),  control  returns  to  Section  3.2,  step  a. 

ERROR:  The  current  SMERFS  software  can  handle  up  to  1000  entries,  which  may  equate 
to  thousands  of  testing  sessions;  the  program  continually  merges  the  testing  times 
until  a  failure  is  observed.  When  the  number  of  entries  exceeds  1000,  the 
message  shown  at  the  bottom  of  the  figure  is  issued  and  control  returns  to 
Section  3.2,  step  a. 


A  RESPONSE  OF  NEGATIVE  VALUES  FOR  THE  PROMPT i 
'ENTER  CPU;  AND  ONE  IP  A  FAILURE  HAPPENED*  OR  ZERO  IF  NOT.” 
WILL  END  THE  PROCESSING. 

ENTER  CPU;  AND  ONE  IF  A  FAILURE  HAPPENED,  OR  ZERO  IF  NOT. 

••THE  SECOND  VALUE  MUST  BE  ZERO  OR  ONE;  RE-ENTER  THAT  VALUE. 

••THE  NUMBER  OF  ENTRIES  EXCEEDS  THE  MAXIMUM  ARRAY  SIZE  FOR  SMERFS. 


FIGURE  3-11.  CPU  DATA  INPUT  PROMPT 


e.  If  both  WC  TBF  and  CPU  TBF  data  are  being  input  (data  type  was  entered  as  a  three),  the 
program  issues  a  message  telling  the  user  how  to  terminate  the  input  sequence  (Figure  3-12). 
As  in  the  case  of  the  WC  keyboard  input,  SMERFS  repeatedly  prompts  for  the  starting  and 
ending  times  of  testing  (in  24-hour  units)  and  the  number  of  failures  observed  in  that  testing 
session.  If  any  software  failures  occurred  in  that  session,  SMERFS  prompts  for  both  die 
failure  time  (again  in  24-hour  units)  and  the  associated  CPU  time  expenditure  for  each 
failure.  SMERFS  then  stores  the  WC  execution  time  in  the  units  as  directed  in  step  a  or  b. 
(No  conversions  are  again  performed  during  the  storage  of  the  CPU  execution  times.)  If  the 
final  failure  was  observed  before  the  end  of  the  testing  session,  SMERFS  prompts  for  the 
expended  CPU  time  between  the  last  failure  and  the  aid  of  the  testing  session.  If  no 
software  failures  were  detected  in  the  session,  SMERFS  prompts  for  the  entire  CPU  time 
expenditure  for  tht  session.  Upon  completion  (as  indicated  through  either  user  direction  or 
error),  control  returns  to  Section  3.2,  step  a. 

ERROR:  The  current  SMERFS  software  can  handle  up  to  1000  entries,  which  may  equate 
to  thousands  of  testing  sessions;  the  program  continually  merges  die  testing  times 
until  a  failure  is  observed.  When  the  number  of  entries  exceeds  1000,  the 
message  shown  at  the  bottom  of  the  figure  is  issued  ami  control  returns  to 
Section  3.2,  step  a. 
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A  RESPONSE  OP  NEGATIVE  VALUES  FOR  THE  PROMPT! 

"ENTER  STARTING  &  ENDING  TIMES;  AND  NUMBER  OP  FAILURES." 

WILL  END  THE  PROCESSING. 

ENTER  STARTING  &  ENDING  TIMES;  AND  NUMBER  OF  FAILURES. 

ENTER  TIME  OF  FAILURE  NUMBER  iii. 

ENTER  ASSOCIATED  CPU  EXPENDED. 

ENTER  LAST  FAILURE'FREE  CPU. 

""THE  NUMBER  OF  ENTRIES  EXCEEDS  THE  MAXIMUM  ARRAY  SIZE  FOR  SMERF8. 


FIGURE  3-12.  WC  AND  CPU  DATA  INPUT  PROMPTS 


d.  If  interval  data  are  being  input  (data  type  was  entered  as  a  four),  the  program  issues  a 
message  telling  the  user  how  to  terminate  the  input  sequence  (Figure  2-13).  It  then 
repeatedly  prompts  for  the  number  of  faults  detected  in  the  testing  interval  and  die  associated 
testing  length  of  that  interval.  No  conversion  or  merging  of  die  input  values  is  performed. 
Upon  completion  (as  indicated  through  either  user  direction  or  error),  control  returns  to 
Section  3.2,  step  a. 

ERROR:  The  current  SMERFS  software  can  handle  up  to  1000  pairs  of  values.  When  die 
number  of  pairs  exceeds  1000,  the  message  shown  at  die  bottom  of  the  figure  is 
issued  and  control  returns  to  Section  3.2,  step  a. 


A  RESPONSE  OF  NEGATIVE  VALUES  FOR  THE  PROMPT: 

"ENTER  FAULT  COUNT  AND  TESTING  LENGTH." 

WILL  END  THE  PROCESSING. 

ENTER  FAULT  COUNT  AND  TESTING  LENGTH. 

•"THE  NUMBER  OF  ENTRIES  EXCEEDS  THE  MAXIMUM  ARRAY  SIZE  FOR  SMERFS. 


FIGURE  3-13.  INTERVAL  DATA  INPUT  PROMPT 


3.2.3  List  the  Current  Data 

If  the  entered  input  option  number  is  three,  SMERFS  generates  one  of  three  possible  reports 
(Figure  3-14).  The  first  report  type  is  for  WC  TBF  or  CPU  TBF  data  (data  type  was  entered  as  a  one 
or  two).  The  columns  of  this  report  contain  consecutive  numbers  for  identification  purposes  and  the  TBF 
data  values,  measured  in  either  WC  or  CPU  units.  The  alphabetic  field  in  the  second  column  will  be  set 
to  the  measurement  units,  and  the  alphabetic  field  appearing  in  the  third  column  is  set  to  "FAILURE- 
FREE"  if  the  last  entered  time  was  not  associated  with  a  software  failure  occurrence.  The  second  report 
type  is  for  WC  TBF  and  CPU  TBF  data  (data  type  was  entered  as  a  three).  This  report  contains  the  same 
information  as  the  first,  but  contains  information  for  both  sets  of  data.  The  third  report  type  is  for 
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interval  data  (data  type  was  uttered  as  a  four)  and  includes  columns  containing  the  interval  number,  the 
number  of  software  faults  occurring  in  that  interval,  and  the  associated  testing  length  of  the  interval. 
Upon  completion,  control  returns  to  Section  3.2,  step  a. 


FIGURE  3-14.  DATA  LIST  OUTPUT 
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CHAPTER  4 
EDIT  MODULE 


4.1  INTRODUCTION 

The  Edit  module  permits  altering  of  the  software  error  data  vectors).  All  data  updates  should  be 
performed  using  die  same  units  of  measurement  as  in  die  data  vector(s).  Through  the  allowed  options, 
the  user  can  obtain  a  listing  of  the  data  and  perform  various  types  of  data  modification.  These 
modifications  may  be  die  result  of  an  incorrectly  entered  dam  value,  a  software  error  being  investigated 
and  subsequently  determined  not  to  be  a  true  software  error,  or  the  omission  of  a  data  value  during  input. 
Once  modified,  the  new  data  vector  may  be  placed  in  the  data  file  Of  *  file  was  generated  as  specified 
in  Chapter  2,  step  g),  or  the  data  values  can  be  retained  locally  without  affecting  die  values  in  the  data 
file.  This  last  option  allows  the  user  to  explore  various  apparent  patterns  appearing  in  the  software  error 
data,  without  necessarily  destroying  the  current  data  file. 


4.2  EXECUTION  FLOW 

This  section  explains  the  prompts  and  inputs  for  the  edit  portion  of  SMERFS.  The  placement  of 
and  die  print  in  die  figures  conform  to  the  strategy  introduced  in  Sections  1.S  and  1.6. 

a.  Edit  Option  Selection 

The  program  prompts  die  user  to  enter  die  type  of  edit  to  be  performed  (Figure  4-1).  If  an 
invalid  option  number  is  entered,  the  same  list  and  second  prompt  are  output. 


ENTER  EDIT  OPTION,  OK  SKKO  FOR  A  LIST. 
THE  AVAILABLE  EDIT  OPTIONS  ARE* 

1  CHANGE  SPECIFIED  ELEMENT 

2  DELETE  SPECIFIED  ELEMEMT(S) 

3  INSERT  UP  TO  10  ELEMENTS 

4  COMBINE  TWO  OR  MORE  ADJACENT  ELEMENTS 

5  CHANGE  THE  TBF  FATALITY  FLAG 

6  LIST  THE  CURRENT  DATA 

7  RETURN  TO  THE  MAIN  PROGRAM 
ENTER  EDIT  OPTION. 


FIGURE  4-1.  EDIT  MENU 
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b.  Change  Specified  Element 

If  the  entered  edit  option  number  is  one,  SMERFS  prompts  for  the  location  number  to  be 
changed  and  the  new  value  to  be  placed  in  that  location  of  the  data  vector  (Figure  4-2).  If 
the  data  type  was  entered  as  a  three  or  four,  SMERFS  also  prompts  for  a  second  value  to 
be  placed  in  the  second  data  vector.  (The  second  vector  contains  the  CPU  TBF  entries  for 
data  type  three,  and  the  interval  testing  lengths  for  data  type  four.)  Upon  completion, 
control  returns  to  step  a. 

WARNING:  If  the  last  entry  of  a  TBF  data  vector  is  changed,  the  reminder  message  shown 
at  the  bottom  of  the  figure  is  issued  prior  to  foe  return. 

ERROR:  If  foe  location  number  of  foe  change(s)  is  in  error,  foe  message  shown  in  foe 

next-to-last  line  of  foe  figure  is  issued  and  control  returns  to  step  a,  leaving 
foe  data  vector(s)  unchanged. 


enter  location  number  and  new  value  for  data  i. 
If  VC  TBF  and  CPU  TBF  data,  or  intaeval  data,  thans 
ENTER  HEM  VALUE  POR  DATA  2. 

End  If 

••THE  EDIT  EXCEEDS  BOUNDS  OF  THE  CURRENT  DATA. 
ENSURE  THE  TBF  FATALITY  FLAG  IS  PROPERLY  SET. 


FIGURE  4-2.  CHANGE  A  SPECIFIED  ELEMENT 


c.  Delete  Specified  Elements) 

If  foe  entered  edit  option  number  is  two,  SMERFS  prompts  for  the  inclusive  starting  and 
ending  location  numbers  to  be  deleted  from  foe  data  vector(s)  (Figure  4-3).  To  delete  a 
single  element,  foe  same  number  should  be  repeated.  Upon  completion,  control  returns  to 
step  a. 

WARNING:  If  foe  end  of  a  TBF  data  vector  is  deleted,  the  reminder  message  shown  at  foe 
bottom  of  the  figure  is  issued  prior  to  the  return. 

ERROR:  If  either  location  number  of  the  deletion  is  in  error,  foe  message  shown  in  foe 

next-to-last  line  of  the  figure  is  issued  and  control  returns  to  step  a,  leaving 
foe  data  vector(s)  unchanged. 
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ENTER  8TARTINO  AMD  ENDING  LOCATIONS  TO  DELETE. 
••THE  EDIT  EXCEEDS  BOUNDS  OF  THE  CURRENT  DATA. 
ENSURE  THE  TBF  FATALITY  FLAG  IS  PROPERLY  SET. 


FIGURE  4-3.  DELETE  SPECIFIED  ELEMENT(S) 


d.  Insert  up  to  10  Elements 

If  die  entered  edit  option  number  is  three,  SMERFS  prompts  for  the  location  number  just 
prior  to  the  insert  and  the  number  of  new  values  (maximum  10)  to  be  placed  in  the  data 
vector(s)  (Figure  4-4).  Hie  location  number  can  be  entered  as  zero  to  cause  insertions  at 
the  beginning  of  the  data.  SMERFS  then  prompts  for  the  new  values  to  be  placed  in  die 
data  vector,  and  will  additionally  prompt  for  a  second  set  of  values  to  be  placed  in  die 
second  data  vector  if  the  data  type  was  entered  as  a  three  or  four.  (The  second  vector 
contains  the  CPU  TBF  entries  for  data  type  three,  and  die  interval  testing  lengths  for  data 
type  four.)  Upon  completion,  control  returns  to  step  a. 

WARNING:  If  die  insertion  is  performed  at  the  end  of  a  TBF  data  vector,  the  reminder 
message  shown  at  the  bottom  of  the  figure  is  issued  prior  to  the  return. 

ERROR:  If  the  location  number  of  die  insertion  is  in  error  or  the  indicated  insertion 

causes  the  number  of  entries  to  exceed  1000,  the  appropriate  message  shown 
in  die  next-to-last  lines  of  the  figure  is  issued  and  control  returns  to  step  a, 
leaving  the  data  vector(s)  unchanged. 


ENTER  LOCATION  NUMBER  JUST  PRIOR  TO  THE  INSERTION  AND  NUMBER  OF 
MEM  ENTRIES. 

ENTER  1  BLEMEMT(S)  FOR  DATA  1 . 

If  VC  TBF  mnd  CPU  TBF  data,  or  interval  data,  thorn 
AMD  1  ELEMENT (S)  FOR  DATA  2. 

Bad  If 

••THE  EDIT  EXCEEDS  BOUNDS  OF  THE  CURRENT  DATA. 

••THE  INSERTION  WILL  EXCEED  THE  MAXIMUM  ARRAY  SIZE  FOR  SMERFS. 

ENSURE  THE  TBF  FATALITY  FLAG  IS  PROPERLY  SET. 


FIGURE  4-4.  INSERT  UP  TO  10  ELEMENTS 
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e.  Combine  Two  or  More  Adjacent  Elements 

If  the  ottered  edit  option  number  is  four,  SMERFS  prompts  for  the  inclusive  starting  and 
ending  location  numbers  that  are  to  be  combined  in  the  data  vector(s)  (Figure  4-5).  Upon 
completion,  control  returns  to  step  a. 

WARNING:  If  the  end  of  a  TBF  data  vector  is  involved  in  the  data  merge,  the  reminder 
message  shown  at  the  bottom  of  the  figure  is  issued  prior  to  die  return. 

ERROR:  If  either  location  number  of  the  data  merge  is  in  error,  the  message  shown  in 

die  next-to-last  line  of  the  figure  is  issued  and  control  returns  to  step  a, 
leaving  the  data  vector(s)  unchanged. 


ENTER  STARTING  AND  ENDING  LOCATIONS  TO  MERGE. 
**THE  EDIT  EXCEEDS  BOUNDS  OF  THE  CURRENT  DATA. 
ENSURE  THE  TBF  FATALITY  FLAG  IS  PROPERLY  SET. 


FIGURE  4-5.  COMBINE  ADJACENT  ELEMENTS 


f.  Change  the  TBF  Fatality  Flag 

If  the  entered  edit  option  number  is  five,  SMERFS  changes  die  value  of  the  fatality  flag  for 
die  final  entry  in  die  TBF  data  vector(s)  (i.e.,  if  the  last  entry  was  recorded  as  being 
terminated  with  the  observation  of  a  software  failure,  the  fatality  flag  is  changed  to  indicate 
that  it  was  not).  The  appropriate  processing  message  shown  in  Figure  4-6  is  then  issued  and 
control  returns  to  step  a. 

ERROR:  If  interval  data  are  resident  (data  type  was  entered  as  a  four),  die  error 

message  shown  at  the  bottom  of  the  figure  is  issued  and  control  returns  to 
step  a,  leaving  the  data  vector(s)  unchanged. 


THE  FATALITY  FLAG  HAS  BEEN  SET  TO  INDICATE  A  FAILURE  OCCURRED 
AT  THE  END  OF  THE  TESTING. 


THE  FATALITY  FLAG  FOR  THE  LAST  DATA  POINT  HAS  BEEN  SET  TO  INDI¬ 
CATE  THAT  A  FAILURE  OCCURRED  FOR  THE  PREVIOUS  DATA  POINT. 


••THIS  OPTION  IS  NOT 


APPLICABLE  FOR  INTERVAL  DATA. 


FIGURE  4-6.  CHANGE  THE  TBF  FATALITY  FLAG 
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g.  List  the  Current  Data 

If  the  entered  edit  option  number  is  six,  SMERFS  generates  one  of  three  possible  reports. 
The  reports  are  the  same  as  those  described  in  Paragraph  3.2.3.  Upon  completion,  control 
returns  to  step  a. 

h.  Return  to  die  Main  Module 

If  die  entered  edit  option  number  is  seven,  the  data  editing  is  over  and  die  user  has  requested 
a  return  to  the  Main  Module  Menu  (Chapter  2,  step  h).  Before  die  return  is  performed, 
SMERFS  determines  if  the  output  data  file(s)  was  opened  during  the  data  input  (Chapter  2, 
step  g)  and  if  any  elements  were  changed.  If  both  tests  showed  true,  SMERFS  allows  the 
user  the  opportunity  to  update  the  WC  TBF,  CPU  TBF,  or  INTERVAL  data  files  to  reflect 
those  changes  by  issuing  the  prompt  shown  in  Figure  4-7.  If  the  update  is  requested,  die 
program  responds  with  the  number  of  elements  stored  and  the  units  of  the  TBF 
measurement.  If  the  data  type  was  entered  as  a  three,  processing  for  both  TBF  data  files 
occurs.  It  is  strongly  advised  that  both  receive  the  same  user  response.  Updating  one  file 
and  not  the  other  may  cause  errors  in  subsequent  attempts  to  analyze  die  two  data  files  in 
a  combined  execution.  Upon  completion,  control  returns  to  the  Main  Module  Menu 
(Chapter  2,  step  h). 


ENTER  ONE  TO  SAVE  THE  NEW  aaaaaaaa  DATA;  ELSE  SERO. 


THE  PILE  STORAGE  OF  ill  aaaaaaaa  ELEMENTS  WAS  PERFORMED. 
THESE  TBF  DATA  ARE  STORED  IN  aaaaaaa 


FIGURE  4-7.  FILE  UPDATE  OPTION  AND  OUTPUT  MESSAGE 
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CHAPTER  5 

UNIT  CONVERSIONS  MODULE 


5.1  INTRODUCTION 

The  Unit  Conversions  module  permits  a  specialized  data  transformation  of  the  TBF  data  vectors). 
Since  the  software  reliability  models  for  execution  time  data  generate  the  estimates  and  predictions  in 
units  analogous  to  those  of  die  stored  failure  history,  it  may  be  appropriate  to  alter  the  storage 
measurements  to  a  different  time  unit.  This  type  of  transformation  is  achieved  through  a  series  of 
multiplications  or  divisions  using  the  following  scale  factors: 

a.  60.0  for  seconds  to  minutes 

b.  60.0  for  minutes  to  hours 

c.  24.0  for  hours  to  days 

d.  07.0  for  days  to  weeks 

e.  04.0  for  weeks  to  months 

f.  12.0  for  months  to  years 

For  example,  if  the  data  are  recorded  in  minutes  and  weeks  are  desired,  then  each  element  of  the  storage 
array  is  divided  by  60  (to  get  the  hours),  divided  again  by  24  (to  get  the  days),  and  then  divided  a  final 
time  by  7  (to  get  the  weeks).  If  more  exact  conversions  are  required,  the  Transformation  Module 
(Chapter  6)  may  be  used. 

Once  modified,  the  new  data  vector  may  be  placed  in  the  data  file  (if  a  file  was  generated  as 
specified  in  Chapter  2,  step  g),  or  the  data  values  can  be  retained  locally  without  affecting  the  values  in 
the  data  file.  This  last  option  allows  the  user  to  obtain  the  model  estimates  and  predictions  in  the  desired 
units,  without  destroying  the  current  data  file.  Experience  has  also  shown  that  die  conversion  of  units 
may  be  a  useful  technique  to  achieve  successful  convergence  for  certain  data  sets  (i.e.,  models  that 
terminate  with  an  error  message  may  converge  when  the  units  are  changed).  If  there  are  large  TBF  data 
values,  considering  time  units  which  are  too  small  (i.e.,  seconds  or  minutes)  may  result  in  problems  of 
overflow  due  to  the  functions  that  are  being  calculated  for  the  various  models.  Larger  time  intervals  (i.e., 
hours  or  days)  may  eliminate  this  problem.  Conversely,  if  die  TBF  data  values  are  small,  using  small 
time  units  may  aid  in  preventing  underflow  of  the  computations. 


5.2  EXECUTION  FLOW 

This  section  explains  the  prompts  and  inputs  for  the  edit  portion  of  SMERFS.  The  placement  of 
and  the  print  in  the  figures  conform  to  the  strategy  introduced  in  Sections  1.5  and  1.6. 
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a.  Data  Type  Selection 

If  WC  TBF  and  CPU  TBF  data  are  resident  (data  type  was  entered  as  a  three),  the  program 
prompts  the  user  to  enter  the  type  of  data  to  be  converted  (Figure  5-1).  If  an  invalid  number 
is  entered,  the  error  message  is  issued  and  the  prompt  is  reissued. 

ERROR:  If  interval  data  are  resident  (data  type  was  entered  as  a  four),  the  error  message 
shown  at  the  bottom  of  the  figure  is  issued  and  control  returns  to  the  Main 
Module  Menu  (Chapter  2,  step  h). 


It  VC  TBF  and  CPU  TBF  data,  than; 

ntm  ONE  FOR  WC  TBF  OR  TWO  FOR  CPU  TBF. 

••DATA  TYPE  ERROR;  TRY  AGAIN  (AFTER  THE  PROMPT). 
Slam,  it  interval  data,  then: 

••THIS  OPTION  IS  NOT  APPLICABLE  FOR  INTERVAL  DATA. 
End  it 


FIGURE  5-1.  DATA  TYPE  PROMPT 


b.  Conversion  Unit  Prompt 

The  program  then  displays  the  units  of  the  stored  data,  lists  die  available  measurement  units, 
and  prompts  the  user  to  enter  the  number  of  the  desired  units  (Figure  5-2). 


THE  Aaaaaaaa  DATA  ARE  CURRENTLY  STORED  IN  aaaaaaa 

THE  DATA  MAY  BE  CONVERTED  TO  THE  FOLLOWING  UNITS: 

1  SECONDS  3  HOURS  5  WEEKS  7  YEARS 

2  MINUTES  4  DAYS  6  MONTHS 

ENTER  MEW  CODE  NUMBER  FOR  THE  DATA. 


FIGURE  5-2.  DESIRED  MEASUREMENT  UNIT  PROMPT 


c.  List  the  Current  Data 

SMERFS  prompts  the  user  to  determine  if  the  listing  (containing  die  converted  data)  is 
desired  (Figure  5-3).  If  the  listing  is  desired,  SMERFS  generates  one  of  two  possible 
reports.  The  reports  are  the  same  as  the  first  two  described  in  Paragraph  3.2.3. 
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ENTER  ONE  TO  LIST  TEE  CURRENT  DATA;  ELSE  ZERO. 


FIGURE  5-3.  DATA  LIST  OPTION 


d.  Return  to  the  Main  Module 

If  die  output  data  file(s)  was  opened  during  the  data  input  (Chapter  2,  step  g),  SMERFS 
allows  the  user  the  opportunity  to  update  the  WC  TBF  or  CPU  TBF  file  (Figure  5-4).  If 
the  file  update  is  desired,  the  program  responds  with  the  number  of  elements  stored  and  the 
(new)  units  of  the  TBF  measurement.  Upon  completion,  control  returns  to  the  Main  Module 
Menu  (Chapter  2,  step  h). 


ENTER  ONE  TO  SAVE  THE  NEW  aaaaaaaa  DATA;  ELSE  ZERO. 

THE  FILE  STORAGE  OF  ill  aaaaaaaa  ELEMENTS  WAS  PERFORMED. 
THESE  TBF  DATA  ARE  STORED  IN  aaaaaaa 


FIGURE  5-4.  FILE  UPDATE  OPTION  AND  OUTPUT  MESSAGE 
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CHAPTER  6 

TRANSFORMATION  MODULE 


6.1  INTRODUCTION 

The  Transformation  module  permits  scaling  of  a  software  error  data  vector.  Five  types  of 
transformations  are  allowed  along  with  the  options  of  restoring  the  data  vector  to  its  non-transformed  state 
and  listing  the  data.  The  Transformation  module  operates  only  on  a  single  data  vector.  Tie  two  TBF 
vectors  cannot  be  transformed  together,  nor  can  the  interval  fault  counts  and  testing  lengths. 
Additionally,  the  transformed  data  are  held  locally.  If  the  retention  of  the  transformed  data  is  desired, 
it  may  be  done  by  entering  the  Edit  module  (Chapter  3),  making  a  dummy  edit  (i.e.,  changing  a 
location’s  value  to  die  current  value),  and  then  allowing  the  Edit  module  to  perform  the  data  file  update. 


6.2  EXECUTION  FLOW 

This  section  explains  die  prompts  and  inputs  for  the  transformation  portion  of  SMERFS.  The 
placement  of  and  the  prim  in  the  figures  conform  to  die  strategy  introduced  in  Sections  1.5  and  1.6. 

a.  Data  Type  Selection 

If  WC  TBF  and  CPU  TBF  data  or  interval  data  are  resident  (data  type  was  entered  as  a  three 
or  four),  SMERFS  prompts  die  user  to  enter  the  type  of  data  to  be  transformed  (Figure  6-1). 
If  an  invalid  number  is  entered,  die  error  message  shown  at  die  bottom  of  foe  figure  is 
issued  and  foe  appropriate  prompt  is  reissued. 


It  VC  TBFmnd  CPU  TBF  date ,  them 

ENTER  ONE  FOR  WC  TBr  OR  TWO  FOR  CTO  TBF. 

Slam,  it  interval  data,  then: 

ENTER  OKEFOR  COUNTS  OR  TWO  FOR  LENGTHS;  THE  RESTORATION  OPTION 
(#6)  Kin.  AFFECT  BOTH  THE  COUNT  AND  LENGTH  DATA. 

End  it 

**DATA  TYPE  ERROR;  TRY  AGAIN  (AFTER  THE  PROMPT) . 


FIGURE  6-1.  DATA  TYPE  PROMPTS 
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b.  Transformation  Option  Selection 

The  program  then  prompts  the  user  to  enter  the  type  of  transformation  to  be  performed 
(Figure  6-2).  If  an  invalid  option  number  is  entered,  the  same  list  and  second  prompt  are 
output. 


ENTER  TRANSFORMATION  OPTION,  OR  ZERO  FOR  A  LIST. 

THE  AVAILABLE  TRANSFORMATION  OPTIONS  ARB: 

1  LOG (A  *  X(I)  +  B)  5  X  *  A 

2  EXP (A  *  X(I)  +  B)  6  RESTORE  THE  DATA 

3  1  **  A  7  LIST  THE  CURRENT  DATA 

41  +  A  8  RETURN  TO  THE  MAIN  PROGRAM 

ENTER  TRANSFORMATION  OPTION. 


FIGURE  6-2.  TRANSFORMATION  MENU 


c.  Transformations  Using  Two  Scale  Factors 


If  the  entered  transformation  option  number  is  one  or  two,  SMERFS  prompts  for  die  two 
transformation  scale  factors  (Figure  6-3).  Upon  completion,  control  returns  to  step  b. 


ERROR:  If,  in  attempting  a  transformation  type  one,  the  log  of  a  non-positive  number  is 
encountered,  the  message  shown  at  the  bottom  of  the  figure  is  issued  and  control 
returns  to  step  b,  leaving  the  data  vector  unchanged. 


ENTER  VALDES  OF  A  AND  B. 

^TRANSFORMATION  ILLEGAL  -  NEGATIVE  OR  ZERO  A  *  X(I)  +  B. 


FIGURE  6-3.  PROMPT  FOR  TWO  SCALE  FACTORS 


d.  Transformations  Using  One  Scale  Factor 

If  die  entered  transformation  option  number  is  three,  four,  or  five,  SMERFS  prompts  for 
die  transformation  scale  factor  (Figure  6-4).  Upon  completion,  control  returns  to  step  b. 

ERROR:  If  a  negative  number  is  encountered  when  attempting  a  transformation  type  three 
with  a  scale  factor  containing  a  decimal  portion,  the  message  shown  at  the  bottom 
of  the  figure  is  issued  and  control  returns  to  step  b,  leaving  die  data  vector 
unchanged. 
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ENTER  VALUE  OP  A. 

“TRANSFORMATION  ILLEGAL  -  NEGATIVE  X(I). 


FIGURE  6-4.  PROMPT  FOR  ONE  SCALE  FACTOR 


e.  Restore  the  Data 

If  the  eiucred  transformation  option  number  is  six,  SMERFS  rewinds  die  appropriate  data 
file  and  enters  the  data  values.  (If  interval  data  are  being  transformed,  restoration  causes 
the  input  of  both  the  fault  counts  and  the  testing  lengths.)  The  program  responds  with  the 
number  of  elements  read  and  the  units  of  the  TBF  measurement  (Figure  6-5).  Upon 
completion,  control  returns  to  step  b. 

ERROR:  If  the  output  data  file  was  not  opened  in  Chapter  2,  step  g,  die  error  message 
shown  at  the  bottom  of  the  figure  is  issued  and  control  returns  to  step  b,  leaving 
the  data  vector  unchanged. 


THE  INPUT  OF  ill  aaaaaaaa  ELEMENTS  WAS  PERFORMED. 

THESE  TBF  DATA  WERE  STORED  IN  aaaaaaa 

*  * RESTORATION  NOT  POSSIBLE;  NEW  DATA  FILES  WERE  NOT  CREATED. 


FIGURE  6-5.  DATA  RESTORATION  MESSAGES 


f.  List  the  Current  Data 

If  the  entered  transformation  option  number  is  seven,  SMERFS  generates  one  of  three 
possible  reports.  The  reports  are  the  same  as  those  described  in  Paragraph  3.2.3.  Upon 
completion,  control  returns  to  step  b. 

g.  Return  to  the  Main  Module 

If  the  entered  transformation  option  number  is  eight,  control  returns  to  the  Main  Module 
Menu  (Chapter  2,  step  h). 
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CHAPTER  7 

GENERAL  STATISTICS  MODULE 


7.1  INTRODUCTION 

The  General  Statistics  module  provides  general  summary  statistics  of  the  software  error  data.  The 
processing  necessary  to  generate  and  output  these  statistics  is  internal  and  requires  no  additional 
information  or  direction  from  die  user. 


7.2  EXECUTION  FLOW 

This  section  explains  die  module  flow  in  the  generation  of  the  statistics  table(s).  The  placement 
of  and  die  print  in  the  figure  conform  to  die  strategy  introduced  in  Sections  1.5  and  1.6.  (Refer  to 
Section  7.3  for  descriptions  of  some  of  die  values  within  these  reports.) 

a.  WC  TBF  or  CPU  TBF 

If  the  data  type  was  entered  as  a  one  or  two,  the  first  report  type  in  Figure  7-1  is  generated 
with  a  WC  TBF  or  CPU  TBF  header.  The  remaining  place  holders  are  replaced  with  die 
indicated  values. 

b.  WC  TBF  and  CPU  TBF 

If  the  data  type  was  entered  as  a  three,  the  first  report  type  is  generated  twice  (once  for  each 
type  of  data). 

c.  Interval  Fault  Counts  with  Equal  Testing  Lengths 

If  the  data  type  was  entered  as  a  four  and  the  testing  lengths  are  the  same  in  all  periods,  the 
second  report  type  is  generated. 

d.  Interval  Fault  Counts  with  Unequal  Testing  Lengths 

If  the  data  type  was  entered  as  a  four  and  the  testing  lengths  are  not  die  same  in  all  periods, 
die  third  report  type  is  generated. 

e.  Return  to  the  Main  Module 

Control  returns  to  the  Main  Module  Menu  (Chapter  2,  step  h). 
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If  VC  TBF  And/or  CPU  TBF  data,  than: 

aaa  TIME-BETWEEN-FAILURES 
RECORDED  IN  aaaaaaa 
WITH  TOTAL  TESTING  TIME  OP  aai 
AND  TIME  OF  LAST  FAILURE  OF  M< 


MEDIAN  OF  THE  DATA 
LOWER  £  UPPER  HINGES 
MINIMUM  AND  MAXIMUM 
NUMBER  OF  ENTRIES 
AVERAGE  OF  THE  DATA 
STD.  DEV.  £  VARIANCE 
SKEWNESS  £  KURTOSIS 


Sima ,  if  intarval  data  (with  aqual  taating  langtha),  than x 

INTERVAL  DATA  WITH  EQUAL  LENGTHS 
WITH  FAULT  COUNTS  TOTALING  TO  iiiiiiiiiii 


MEDIAN  OF  THE  DATA  * 
LOWER  £  UPPER  HINGES  * 
MINIMUM  AND  MAXIMUM  * 
NUMBER  OF  ENTRIES  * 
AVERAGE  OF  THE  DATA  * 
STD.  DEV.  £  VARIANCE  * 
SKEWNESS  £  KURTOSIS  * 


Sima,  if  intarval  data  ( with  unequal  taating  langtha),  than: 

INTERVAL  DATA  WITH  VARYING  LENGTHS 


MEDIAN  OF  THE  DATA  * 
LOWER  fi  UPPER  HINGES  * 
MINIMUM  AND  MAXIMUM  * 


MEDIAN  OF  THE  DATA  * 
LOWER  £  UPPER  HINGES  * 
MINIMUM  AND  MAXIMUM  • 


COUNT  £  LENGTH  TOTALS* 
RATIO  OF  THE  PRIOR  * 
NUMBER  OF  ENTRIES  * 


End  if 


-  FAULT  COUNTS 

eeeec 
eeeeeeeeeeeeee 
eeeeeeeeeeeeee 

-  INTERVAL  LENGTHS  — — 

eeeec 
eeeeeeeeeeeeee 
eeeeeeeeeeeeee 

-  COMBINATION  - 

eeeeeeeeeeeeee  eeeeeeeeeeeeee 


ill 


FIGURE  7-1.  GENERAL  STATISTICS  REPORTS 


7.3  VALUES  DESCRIPTIONS 

The  median  of  the  data  (Figure  7-1)  is  a  measure  of  central  tendency.  It  represents  the  value  such 
that  SO  percent  of  the  data  have  values  below  it;  analogously,  SO  percent  of  the  values  are  greater. 
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The  lower  and  upper  hinges  represent  a  breakup  of  each  of  the  two  sections  of  the  data,  determined 
by  the  median,  into  two  equal  parts.  Thus,  the  first  value  of  the  hinge  has  23  percent  of  the  data  points 
being  smaller  (75  percent  therefore  being  larger)  and  the  second  has  73  percent  being  smaller  (25  percent 
therefore  being  larger).  They  both  are  a  measure  of  the  spread  of  the  data  (Reference  3). 

The  minimum  and  maximum  values  are,  respectively,  the  smallest  and  largest  values  in  the  data 
base;  the  number  of  entries  simply  shows  the  number  of  points  in  the  data  base. 

For  the  first  two  report  types,  the  summary  statistics  also  include  die  average,  standard  deviation, 
sample  variance,  skewness,  and  kurtosis.  The  skewness  is  a  measure  of  the  symmetry  of  the  sample  and 
the  kurtosis  indicates  how  "peaked"  die  sample  is. 

For  the  third  report  type,  the  summary  statistics  include  the  total  number  of  faults  found,  the  total 
amount  of  testing  time,  the  ratio  of  the  two  totals  (showing  an  estimate  for  the  number  of  faults  detected 
per  testing  period),  and  the  number  of  points  in  the  data  base. 
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CHAPTERS 

RAW  DATA  PLOT  MODULE 


8.1  INTRODUCTION 

The  Raw  Data  Plot  module  generates  plots  of  die  software  error  data.  The  term  “raw  data*  is  used 
to  show  that  the  plot  is  of  die  actual  data  rather  than  data  from  a  software  reliability  prediction.  The  plots 
are  the  result  of  an  internal  line  printer  plotter.  The  internal  plotter  produces  very  crude  graphs  to  assist 
the  user  in  quick  interactive  examination  of  the  data.  It  is,  however,  highly  recommended  that  users 
make  use  of  die  optional  Plot  file  (Chapter  2,  step  d)  to  produce  high-quality  plots.  Examples  of  this  file 
can  be  found  in  Appendixes  C  and  D. 


8.2  EXECUTION  FLOW 

This  section  explains  die  prompts  and  input  for  the  raw  data  plot  portion  of  SMERFS.  The 
placement  of  and  die  print  in  die  figures  conform  to  die  strategy  introduced  in  Sections  1.3  and  1.6. 

a.  Dau  Type  Selection 

If  WC  TBF  and  CPU  TBF  data  are  resident  (data  type  was  entered  as  a  three),  die  program 
prompts  die  user  to  otter  die  type  of  data  to  be  plotted  (Figure  8-1).  If  an  invalid  number 
is  entered,  die  message  shown  at  the  bottom  of  die  figure  is  issued  and  die  prompt  is 
reissued. 


It  VC  TBF  and  CPO  TBF  dace,  then* 

moot  ONE  FOR  ¥C  TBF  OK  TWO  POR  CTO  TBF. 

ERROR;  THY  AGAIN  (AFTER  THE  PROMPT) . 

fold  if 


FIGURE  8-1.  DATA  TYPE  PROMPT 


b.  Tide  Prompts 

SMERFS  then  prompts  for  die  plot  tide  for  either  the  execution  TBF  data  or  die  interval 
fault  count  data  (Figure  8-2).  SMERFS  is  designed  to  generate  a  second  plot  of  the  interval 
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testing  lengths;  however,  the  title  for  that  plot  is  not  prompted  until  it  is  determined  if  the 
user  desires  the  plot  to  be  generated  (step  e).  For  correct  centering  on  the  plot,  the  title(s) 
should  be  centered  in  the  30-character  field. 


c.  Smoothing  Prompt 

If  die  sample  size  is  greater  than  six,  SMERFS  prompts  the  user  to  determine  if  smoothing 
of  the  WC  DATA,  CPU  DATA,  or  COUNT  DATA  is  desired  (Figure  8-3).  The  smoothing 
technique  is  described  in  Reference  7. 


CUTER  OMB  TO  SMOOTH  THE 


ELSE  EERO. 


FIGURE  8-3.  SMOOTHING  PROMPT 


d.  Plot  Generations 

The  plot  is  then  generated  (Figure  8-4),  including  the  user-specified  title  and  die  program- 
determined  axis  labels.  The  units  for  foe  TBF  measurement  will  also  be  placed  on  foe  plot 
as  a  second  y-axis  label. 

The  plotter  can  only  reflect  a  maximum  of  SO  elements  on  foe  x-axis.  The  message  shown 
in  foe  last  line  of  Figure  8-4  not  only  provides  for  a  pause  between  plots,  but  also  allows 
for  termination  of  foe  plotting. 

NOTE;  The  minimum  and  maximum  values  for  foe  y-axis  are  defined  over  all  values  to 
be  plotted,  not  just  those  reflected  in  foe  plot  of  SO  points. 
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ill  111  Ui~~  ill  ill  ill 

uiuauu 

ENTER  ONE  TO  VIEW  TEE  NEXT  SECTION;  ELSE  SERO  TO  EXIT  PLOTTER. 


FIGURE  8-4.  PLOT  FORMAT  AND  PROMPTS 


e.  Length  Plot  Prompts 

If  interval  data  are  resident  (data  type  was  entered  as  a  four),  the  program  prompts  the  user 
to  determine  if  the  plot  of  the  interval  testing  lengths  is  desired  (Figure  8-5).  This  option 
exists  because  the  lengths  are  usually  set  to  one;  and  therefore,  the  plot  is  not  needed.  If 
an  affirmative  response  is  made,  then  the  title  line  is  prompted  and  a  plot  similar  to  that 
shown  in  Figure  8-4  is  generated.  (The  second  y-axis  label  is  set  to  blanks  for  the  interval 
plot.) 


ENTER  ONE  FOR  ASSOCIATED  LENGTH  PLOT;  ELSE  ZERO. 
ENTER  A  LENGTH  PLOT  TITLE  (UP  TO  30  CHARACTERS). 


FIGURE  8-5.  LENGTH  PLOT  PROMPTS 


f.  Return  to  the  Main  Module 

After  the  plotting,  the  routine  automatically  returns  control  to  the  Main  Module  Menu 
(Chapter  2,  step  h). 
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CHAPTER  9 

MODEL  APPLICABILITY  MODULE 


9.1  INTRODUCTION 

The  Model  Applicability  module  performs  the  applicability  analyses  as  proposed  by  Bev  Littlewood 
(Reference  5).  Quoting  the  abstract  of  that  reference: 

"Different  software  reliability  models  can  produce  very  different  answers  when  called 
upon  to  predict  future  reliability  in  a  reliability  growth  context.  Users  need  to  know  which, 
if  any,  of  the  competing  predictions  are  trustworthy.  Some  techniques  are  presented  which 
form  die  basis  of  a  partial  solution  to  this  problem.  In  addition,  it  is  shown  that  this 
approach  can  point  the  way  towards  more  accurate  predictions  via  models  which  learn  from 
past  behaviour." 

All  four  analyses  [Accuracy  (Prequential  Likelihood),  Bias,  Noise,  and  Trend]  have  been  implemented 
for  the  execution  time  models.  Only  the  Accuracy  has  been  implemented  for  die  interval  data  models. 


9.2  EXECUTION  FLOW 

This  section  explains  the  prompts  and  inputs  for  die  model  applicability  portion  of  SMERFS.  The 
placement  of  and  the  print  in  the  figures  conform  to  die  strategy  introduced  in  Sections  1.5  and  1.6. 

a.  Data  Type  Selection 

If  WC  TBF  and  CPU  TBF  data  are  resident  (data  type  was  entered  as  a  three),  the  program 
prompts  the  user  to  enter  the  type  of  data  to  be  analyzed  (Figure  9-1).  If  an  invalid  number 
is  entered,  the  error  message  shown  at  die  bottom  of  die  figure  is  issued  and  the  prompt  is 
reissued. 


then* 

ENTER  OKB  FOR  WC  TBF  OR  TWO  FOR  CPU  TBF, 

••DATA  TYPE  ERROR;  TRY  AGAIN  (AFTER  THE  PROMPT), 
Encf  if 


FIGURE  9-1.  DATA  TYPE  PROMPT 
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b.  Analysis  Range  Specification 

SMERFS  then  indicates  the  range  over  which  each  analysis  will  take  place  (Figure  9-2). 
The  limits  are  set  to  one-half  and  one  less  than  the  number  of  failures  for  execution  time 
data.  Similarly,  the  limits  are  set  to  one-half  and  one  less  than  the  number  of  periods  for 
interval  data.  If  the  values  are  acceptable,  enter  a  zero.  Otherwise,  if  new  limits  are 
desired,  then  an  affirmative  response  should  be  made  and  the  program  issues  the  prompts 
for  the  new  limits  (Figure  9-2). 

ERROR:  If  the  resulting  range  is  too  small,  the  message  shown  at  the  bottom  of  the  figure 
is  issued  and  control  returns  to  the  Main  Module  Menu  (Chapter  2,  step  h). 


THE  ANALYSIS  OF  THE  DATA  SET  OP  SIZE  ill  WILL  (BY  DEFAULT)  OCCUR 
BETWEEN  ELEMENTS  ill  AND  ill.  ENTER  ONE  TO  USE  A  DIFFERENT  RANGE 
OR  EBRO  TO  USB  THIS  RANGE  OF  ill  ELEMENTS. 

If  uamr  ovmrridm  v am  malmctmd ,  t ham 

ENTER  STARTING  INDEX,  GREATER  TRAN  1. 

ENTER  ENDING  INDEX,  LESS  TRAN  iii. 

End  if 

** ANALYSIS  CANNOT  BE  MADE;  RANGE  MUST  BE  AT  LEAST  TWO. 


FIGURE  9-2.  ANALYSIS  RANGE  SPECIFICATION 

c.  Transfer  to  the  Indicated  Analysis 

SMERFS  then  transfers  control  to  Paragraph  9.2. 1  if  execution  time  data  are  resident,  or 
Paragraph  9.2.2  if  interval  data  are  resident. 


9.2.1  Applicability  Analysis  for  Execution  Time  Data 

If  execution  time  data  are  resident  (data  type  was  entered  as  a  one,  two,  or  three),  then: 

a.  The  program  prompts  the  user  to  enter  the  type  of  analysis  to  be  performed  (Figure  9-3). 
If  an  invalid  option  number  is  entered,  the  same  list  and  second  prompt  are  output.  (Refer 
to  Section  9.3  for  descriptions  of  these  analyses.) 

NOTE:  During  the  execution  of  these  analyses,  it  is  not  necessary  to  record  die  values 
for  the  various  statistics,  as  a  summary  table  of  those  values  will  be  scripted 
when  an  eight  is  entered  for  the  model  applicability  option  number. 
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BURK  MODEL  APPLICABILITY  ANALYSIS  OPTION,  (Ml  SEBO  PON  A  LIST. 
THE  AVAILABLE  MODEL  APPLICABILITY  ANALYSIS  OPTIONS  ARE: 

1  THE  MODEL  ACCURACY  STATISTIC 

2  THE  MODEL  ACCURACY  STATISTIC  AND  SCATTER  PLOT 

3  THE  MODEL  BIAS  STATISTIC  AND  U-PLOT 

4  THE  MODEL  BIAS  STATISTIC,  U-PLOT,  AND  SCATTER  PLOT 

5  THE  MODEL  NOISE  STATISTIC 

6  THE  MODEL  TREND  STATISTIC  AND  Y-PLOT 

7  PULL  ANALYSIS  WITH  SUMMARY  REPORT  ONLY  PRINTOUT 

8  RETURN  TO  THE  MAIN  PROGRAM 

ENTER  MODEL  APPLICABILITY  ANALYSIS  OPTION. 


FIGURE  9-3.  EXECUTION  TIME  MODEL  APPLICABILITY  MENU 


b.  The  program  then  prompts  for  a  flag  indicating  if  all  execution  time  models  are  to  be 
executed  (Figure  9-4).  If  the  execution  of  all  models  is  desired,  enter  zero.  Otherwise,  if 
die  selection  of  certain  models  is  desired,  then  an  affirmative  response  should  be  made  and 
die  program  will  allow  the  user  to  chose  the  models  to  be  executed. 


FIGURE  9*4.  EXECUTION  TIME  MODEL  SELECTION  PROMPTS 


c.  If  the  entered  model  applicability  option  number  is  one,  SMERFS  outputs  the  computed 
statistics  for  the  Accuracy  of  the  model  predictions  (along  with  alphabetic  labels  to  identify 
the  models)  (Figure  9-5).  Upon  completion,  control  returns  to  step  a. 

ERROR:  If  any  execution  of  the  model  in  the  range  of  iterations  defined  in  Section  9.2, 
step  b  failed  to  achieve  the  convergence  criteria,  die  analysis  for  that  model  is 
ended  and  the  error  message  shown  at  the  bottom  of  die  figure  is  issued. 
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THE  FOLLOWING  WERE  DETERMINED  BASED  ON  ill  POINTS 
Do  for  *<e h  of  the  specified  models s 
It  the  execution  wee  successful,  them 

maaauaaaauaaaauauaaaa  STATISTIC  IS  aaaatMttaaaaa 
Else,  if  the  execution  wes  not  successful,  them 

**a*aaaaaaaaaaaaaaaaaaaaaaaaa  FAILED  RFLAG  i  ON  ITER,  ill  <OBS.  ill) 
End  it 
End  do 


FIGURE  9-5.  MODEL  ACCURACY  STATISTIC  PRINTOUT 

d.  If  the  entered  model  applicability  option  number  is  two,  SMERFS  performs  the  same 
processing  as  described  in  step  c;  however,  the  individual  values  used  to  define  the  statistic 
are  also  plotted  (Figure  9-6).  Upon  completion,  control  returns  to  step  a. 

The  plotter  can  only  reflect  a  maximum  of  SO  elements  on  the  x-axis.  The  messages  shown 
at  the  bottom  of  the  figure  not  only  provide  for  a  pause  between  plots,  but  also  allow  for 
termination  of  the  plotting. 


NOTE:  The  minimum  and  maximum  values  for  the  y-axis  are  defined  over  all  values  to 

be  plotted,  not  just  those  reflected  in  the  plot  of  SO  points. 


FIGURE  9-6.  MODEL  ACCURACY  SCATTER  PLOT 
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e.  If  the  entered  model  applicability  option  number  is  three,  SMERFS  outputs  the  computed 
statistics  for  the  Bias  of  the  model  predictions  and  the  corresponding  U-Plot  (along  with 
alphabetic  labels  to  identify  the  models)  (Figure  9-7).  Upon  completion,  control  returns  to 
step  a. 

ERROR:  If  any  execution  of  the  model  in  the  range  of  iterations  defined  in  Section  9.2, 
step  b  failed  to  achieve  the  convergence  criteria,  the  analysis  for  that  model  is 
ended  and  the  error  message  shown  at  the  bottom  of  the  figure  is  issued. 


Do  for  each  of  the  specified  models: 

If  the  execution  was  successful,  then: 

U— PLOT  FOR  aaaaaaaaaaaaaaaaaaaaaaa&aaa 
KOLMOGOROV  DISTANCE  IS  eeeeaeaeeeeeae 
BASED  ON  ill  OF  ill  POINTS 

1  I 

i  *  *  0,0  to  1,1  line 

I  U  *  computed  value 

I  $  *  *  end  U  too  close  to  distinguish  on  plot 


0  1/NS  WITH  A  STEP  OF  i 

ENTER  ONE  TO  CONTINUE  THE  PROCESSING. 

Else ,  if  the  execution  was  not  successful,  then : 

**aaaaaaaaaaaaaaaaaaaaaaaaaaa  FAILED  RFLAG  i  ON  ITER,  iii  (OBS.  ill) 
End  if 
End  do 


FIGURE  9-7.  MODEL  BIAS  STATISTIC  PRINTOUT  AND  U-PLOT 


f.  If  the  entered  model  applicability  option  number  is  four,  SMERFS  performs  the  same 
processing  and  displays  the  same  data  as  described  in  step  e;  however,  the  unsorted  values 
from  the  Y-Plot  are  also  plotted  through  a  plot  similar  to  that  described  in  step  d.  Upon 
completion,  control  returns  to  step  a. 


9-5 


NSWCDD  TR  84-373 

g.  If  the  entered  model  applicability  option  number  is  five,  SMERFS  outputs  the  computed 
statistics  for  the  Noise  of  the  model  predictions  using  the  same  report  format  as  shown  in 
Figure  9-5.  Upon  completion,  control  returns  to  step  a. 


FIGURE  9-8.  MODEL  TREND  STATISTIC  PRINTOUT  AND  Y-PLOT 


ERROR:  If  any  execution  of  the  model  in  the  range  of  iterations  defined  in  Section  9.2, 
step  b  failed  to  achieve  the  convergence  criteria,  the  analysis  for  that  model  is 
ended  and  the  error  message  shown  at  the  bottom  of  the  figure  is  issued. 


h.  If  foe  entered  model  applicability  option  number  is  six,  SMERFS  outputs  foe  computed 
statistics  for  foe  Trend  of  foe  model  predictions  and  foe  corresponding  Y-Plot  (along  with 
alphabetic  labels  to  identify  the  models)  (Figure  9-8).  Upon  completion,  control  returns  to 
step  a. 


ERROR:  If  any  execution  of  foe  model  in  foe  range  of  iterations  defined  in  Section  9.2, 
step  b  failed  to  achieve  foe  convergence  criteria,  foe  analysis  for  that  model  is 
ended  and  foe  error  message  shown  at  foe  bottom  of  foe  figure  is  issued. 


Do  for  each  of  the  specif  led  models: 

If  the  execution  was  successful ,  then: 


Y-PLOT  FOR  aaaaaaaaaaaaaaaaaaaaaaaaaaa 

KOLMOGOROV  DISTANCE  IS  eeeaeaeaeooaaa 
BASED  ON  ill  OF  iii  POINTS 


*  -  0,0  to  1,1  line 
Y  m  computed  value 

$  -  *  and  Y  too  close  to  distinguish  on  plot 


0  1/NS  WITH  A  STEP  OF  i 

ENTER  ONE  TO  CONTINUE  THE  PROCESSING. 

Else,  if  the  execution  was  not  successful,  then: 


FAILED  RFLAG  i  ON  ITER.  Hi  (OBS.  ill) 


I 

i 
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i.  If  the  entered  model  applicability  option  number  is  seven,  SMERFS  computes  die  statistics 
for  the  Accuracy,  Bias,  Noise,  and  Trend  of  the  model  predictions  for  each  selected  model; 
however,  the  only  output  are  lines  indicating  which  of  the  four  analyses  is  being  performed 
and  which,  if  any,  of  the  models  failed  to  achieve  the  convergence  criteria  (Figure  9-9).  No 
printout  of  the  successful  models  will  be  listed.  Upon  completion,  control  continues  with 
step  j. 

NOTE:  Since  the  convergence  criteria  are  the  same  for  all  four  analyses,  models  which 

failed  for  the  first  analysis  will  not  be  attempted  for  the  other  three  analyses. 


CURRENTLY  WORKING  ON  THE  MODEL  ACCURACY  ANALYSIS. 

It  the  execution  was  not  successful,  them 
*  *  aeaaaaaaaaaaaaaaaaaaaaaaa  aa  FAILED  RFLAG  i  ON  ITER,  ill  (OBS.  ill) 
snd  it 

CURRENTLY  WORKING  ON  THE  MODEL  BIAS  ANALYSIS. 

It  the  execution  from  the  eccuracy  analysis  was  not  successful ,  them 
**&&aaaaaaaaaaaaaaaaaaaaaaaaa  SKIPPED  BECAUSE  OF  PRIOR  FAILURE 
Snd  if 

CURRENTLY  WORKING  ON  THE  MODEL  NOISE  ANALYSIS. 

Zf  the  execution  from  the  accuracy  analysis  was  not  successful,  them 
••aaaaaaaaaaaaaaaaaaaaaaaaaaa  SKIPPED  BECAUSE  OF  PRIOR  FAILURE 
Snd  it 

CURRENTLY  WORKING  ON  THE  MODEL  TREND  ANALYSIS. 

If  the  execution  from  the  accuracy  analysis  was  not  successful,  them 
**aaaaaaaaaaaaaaaaaaaaaaaaaaa  SKIPPED  BECAUSE  OF  PRIOR  FAILURE 
End  if 


FIGURE  9-9.  FULL  APPLICABILITY  ANALYSIS  MESSAGES 


j.  If  the  entered  model  applicability  option  number  is  eight  (or  if  the  option  was  entered  as  a 
seven  and  the  processing  is  completed),  SMERFS  computes  the  rankings  of  the  models  for 
each  of  the  analyses,  as  well  as  the  overall  ranking  (based  on  a  linear  average  of  all  of  the 
analyses).  A  table  of  the  results  is  then  output  (Figure  9-10)  and  control  returns  to  the  Main 
Module  Menu  (Chapter  2,  step  h). 
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THE  FOLLOWING  TABLE  SUMMARIZES  THE  PERFORMED  MODEL  APPLICABILITY 
ANALYSIS.  ZEROS  UNDER  THE  VALUES  PORTION  INDICATE  THE  MODEL  WAS 
EITHER  NOT  RUN  OR  FAILED  PRIOR  TO  COMPLETION.  THE  INTEGER  NUMBER 
TO  THE  RIGHT  OF  THE  VALUE  REFLECTS  THE  RANKING.  THE  VALUE  TO  THE 
FAR  RIGHT  REPRESENTS  THE  AVERAGE  RANKING  OF  EACH  MODEL. 

MODEL  ACCURACY/RANK  BIAS/RANK  NOISE/RANK  TREND/RANK  AVG 

mibbb  m-mmmmmmmmmmmm  mmmmmmmmmmmmm  mmmmmmxmmxmmm  mmmmmmm mmmmmm  mmm 

GEO  •eeeaeaeeee  i  eeeeeeeeeee  i  — ee — ee—  i  aaaaaeaaaaa  i  fff 
JAM  aaaaaaaaaaa  i  aaaaeaaeaaa  i  aeaaaaeaaaa  i  aaaaaaaaaaa  i  fff 
LAV-L  aaaaaaaaaaa  i  aaaaaaaaaaa  i  aaaaaaaaaaa  1  aaaaaaaaaaa  i  fff 
LAV-Q  aaaaaeeeaae  i  aeaeeaeeeae  i  eeeeeeeeeee  i  aaaaaaaaaaa  i  fff 
HUS-B  aaeeeeeeeee  i  eeeeeeeeeee  i  aeaaaaeeeea  i  eeeeeeeeeee  i  fff 
MUS-L  eeeeeeeeeee  i  eeeeeeeeeee  i  eeeeeeeeeee  i  aeeeeeaaaaa  i  fff 
NPT  aaeeeeeeeee  i  eeeeeeeeeee  i  eeeeeeeeeee  i  eeeeeeeeeee  1  fff 


FIGURE  9-10.  EXECUTION  TIME  APPLICABILITY  SUMMARY  TABLE 


9.2.2  Applicability  Analysis  for  Interval  Data 

If  interval  data  are  resident  (data  type  was  entered  as  a  four),  then: 

a.  The  program  prompts  the  user  to  enter  the  type  of  analysis  to  be  performed  (Figure  9-1 1). 
If  an  invalid  option  number  is  entered,  the  same  list  and  second  prompt  are  output.  (Refer 
to  Section  9.3  for  descriptions  of  these  analyses.) 

NOTE:  During  the  execution  of  these  analyses,  it  is  not  necessary  to  record  the  values 

for  the  various  statistics,  as  a  summary  table  of  those  values  will  be  scripted 
when  a  four  is  entered  for  the  model  applicability  option  number. 


ENTER  MODEL  APPLICABILITY  ANALYSIS  OPTION,  OR  ZERO  FOR  A  LIST. 
THE  AVAILABLE  MODEL  APPLICABILITY  ANALYSIS  OPTIONS  ARE* 

1  THE  MODEL  ACCURACY  STATISTIC 

2  THE  MODEL  ACCURACY  STATISTIC  AND  SCATTER  PLOT 

3  SUMMARY  REPORT  ONLY  PRINTOUT 

4  RETURN  TO  THE  MAIN  PROGRAM 

ENTER  MODEL  APPLICABILITY  ANALYSIS  OPTION. 


FIGURE  9-11.  INTERVAL  DATA  MODEL  APPLICABILITY  MENU 


b.  The  program  then  prompts  for  a  flag  indicating  if  all  interval  data  models  are  to  be  executed 
(Figure  9-12).  If  the  execution  of  all  models  is  desired,  enter  zero.  Otherwise,  if  the 
selection  of  certain  models  is  desired,  then  an  affirmative  response  should  be  made  and  the 
program  will  allow  the  user  to  chose  the  models  to  be  executed. 
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ENTER  ONE  TO  SELECT  THE  MODELS,  OR  ZERO  TO  RUN  ALL  SIX. 

It  user  specification  was  selected,  then t 

ENTER  ONE  FOR  BROOKS  AND  MOTLEY  BINOMIAL  MODEL;  ELSE  ZERO. 
ENTER  ONE  FOR  BROOKS  AND  MOTLEY  POISSON  MODEL;  ELSE  ZERO. 
ENTER  ONE  FOR  GENERALIZED  POISSON  MODEL;  ELSE  ZERO. 

ENTER  ONE  FOR  NON-HOMOGENEOUS  POISSON  MODEL;  ELSE  ZERO. 
ENTER  ONE  FOR  SCHNEI DEWIND  MODEL;  ELSE  ZERO. 

ENTER  ONE  FOR  S-SKAPED  RELIABILITY  GROWTH  MODEL;  ELSE  ZERO. 

End  if 


FIGURE  9-12.  INTERVAL  DATA  MODEL  SELECTION  PROMPTS 


c.  If  either  of  the  Brooks  and  Motley  models  or  the  Generalized  Poisson  model  were  selected 
to  be  executed,  the  additional  values  as  shown  in  Figure  9-13  are  prompted. 


It  either  of  the  Brooks  and  Motley  models  was  selected ,  them 

ENTER  ALPHA  (THE  PROB.  OF  CORRECTING  FAULTS,  WITHOUT  INSERTING 
NEW  ONES)  FOR  THE  BROOKS  AND  MOTLEY  MODEL. 

End  if 

If  the  Generalized  Poisson  model  was  selected,  then: 

ENTER  ALPHA  (COUNT(I) ** ALPHA)  FOR  THE  GENERALIZED  POISSON  MODEL. 
End  if 


FIGURE  9-13.  INTERVAL  DATA  ADDITIONAL  PROMPTS 


d.  If  the  entered  model  applicability  option  number  is  one,  SMERFS  outputs  the  computed 
statistics  for  the  Accuracy  of  the  model  predictions  using  the  same  report  format  as  described 
in  Paragraph  9.2.1,  step  c.  Upon  completion,  control  returns  to  step  a. 

ERROR:  If  any  execution  of  the  model  in  the  range  of  iterations  defined  in  Section  9.2, 
step  b  failed  to  achieve  the  convergence  criteria,  the  analysis  for  that  model  is 
ended  and  the  error  message  shown  at  the  bottom  of  Figure  9-5  is  issued. 

e.  If  the  entered  model  applicability  option  number  is  two,  SMERFS  outputs  the  computed 
statistics  for  the  Accuracy  of  the  model  predictions  and  the  additional  plot  of  the  values  used 
to  define  the  statistic  using  the  same  report  format  as  described  in  Paragraph  9.2.1,  step  d. 
Upon  completion,  control  returns  to  step  a. 
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f.  If  the  entered  model  applicability  option  number  is  three,  SMERFS  computes  the  statistics 
for  the  Accuracy  of  the  model  predictions;  however,  the  only  output  are  lines  indicating  that 
the  analysis  is  being  performed  and  which,  if  any,  of  the  models  failed  to  achieve  the 
convergence  criteria  (the  first  block  of  Figure  9-9).  No  printout  of  the  successful  models 
will  be  listed.  Upon  completion,  control  continues  with  step  g. 

g.  If  the  entered  model  applicability  option  number  is  four  (tv  if  the  option  was  entered  as  a 
three  and  the  processing  is  completed),  SMERFS  computes  the  rankings  of  die  models  for 
die  analysis.  A  table  of  the  results  is  then  output  (Figure  9-14)  and  control  returns  to  the 
Main  Module  Menu  (Chapter  2,  step  h). 


THK  FOLLOWING  TABLE  SUMMARIZES  THE  PERFORMED  MODEL  APPLICABILITY 
ANALYSIS,  ZEROS  UNDER  THE  VALUES  PORTION  INDICATE  THE  MODEL  WAS 


EITHER  NOT  RUN  OR  FAILED  PRIOR  TO  COMPLETION.  THE  INTEGER  NUMBER 
TO  THE  RIGHT  OF  THE  VALUE  REFLECTS  THE  RANKING* 


MODEL  ACCURACY/RANK 


BAM-B  MmemH  i 
BAM-P  MMteMMt 
CPU  — — — —  i 
MPI  — — — — —  4 

SSH  MMSHMM  i 


FIGURE  9-14.  INTERVAL  DATA  APPLICABILITY  SUMMARY  TABLE 


9.3  ANALYSIS  OUTPUTS  AND  CONSIDERATIONS 

The  bases  of  the  four  methods  are  described  in  detail  in  References  5  and  6.  For  the  Accuracy 
analysis,  one  is  looking  for  the  "best”  model  in  forecasting  future  values  of  the  time  to  next  failure  using 
the  past  failures  and  the  negative  of  the  prequential  likelihood  function.  The  bigger  this  number  is,  the 
"higher"  the  likelihood  is  that  the  future  data  points  came  from  a  given  model’s  probability  density 
function.  Thus,  the  model  with  the  largest  number  appears  "best"  using  this  criteria. 

The  Bias  plot  is  looking  for  departures  of  the  model  in  terms  of  underpredicting,  overpredicting, 
or  some  functional  combination  of  the  two  for  future  values  of  the  time  to  next  failure.  If  there  is  no 
’bias"  in  the  model,  the  U;’s  should  be  uniformly  distributed  over  the  interval  zero  to  one;  and  hence, 
the  U-plot  should  show  the  values  falling  along  the  line  y-x.  The  Kolmogorov  distance  is  a  measure 
of  die  discrepancy  between  the  U-values  and  this  line.  If  this  statistic  is  too  large  in  relationship  to  die 
value  taken  from  a  Kolmogorov-Smimov  (K-S)  one  sample  table,  it  could  be  concluded  that  die  U/s  are 
not  random.  Thus,  the  model  would  then  appear  to  have  a  bias  within  it  for  prediction  purposes.  If  die 
Uj’s  are  consistendy  above  the  line  y=x,  the  model  is  overpredicting  the  times  to  next  failures. 
Conversely,  if  the  Ui’s  are  consistently  below  the  line,  the  predictions  are  too  low.  For  this  criteria,  one 
looks  for  models  with  no  Bias  (i.e.,  that  pass  the  K-S  test  for  uniformity). 
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In  some  models,  the  predictions  may  be  too  high  (low)  early  on  and  then  too  low  (high)  in  later 
stages.  This  will  cause  the  model  to  appear  to  be  "unbiased'  using  the  U-piots.  The  Trend  plot  (Y-ploc) 
eliminates  this  problem  as  it  looks  also  at  the  behavior  of  the  statistics  over  time.  The  Trend  statistics 
show  die  consistency  of  the  model’s  Bias.  A  small  value  means  the  model  is  better  able  to  adapt  to 
changes  in  the  behavior  of  the  data;  and  hence,  will  tend  to  yield  better  prediction  performance.  Again, 
one  is  looking  for  the  model  whose  Y-plot  shows  the  ^-values  falling  along  the  line  y=x  and  pass  the 
K-S  test  for  uniformity. 

The  Noise  statistic  based  on  the  forecasted  medians  of  die  failure  time  distribution  is  used  to 
indicate  the  model  giving  the  least  variable  forecasts.  The  model  having  the  smallest  noise  is  thus  the 
"most  stable"  for  a  given  particular  data  set. 

The  user  may  prefer  any  one  or  combination  of  the  above  criteria  to  aid  in  determining  candidate 
models  for  analysis.  The  authors  warn  that  if  a  particular  criteria  is  used  be  sure  to  consider  not  only 
the  ranking  provided  by  the  SMERFS  program,  but  also  the  magnitude  of  die  statistics  computed.  There 
may  be  little  difference  in  die  magnitude  between  the  model  ranked  number  one  and  the  lowest  ranked 
one  using  a  chosen  criteria.  The  model  applicability  analysis  is  intended  as  only  a  preliminary  filtering 
of  the  models.  The  selected  models  must  then  be  further  analyzed  using  the  goodness-of-fit  criteria 
(Chapter  12)  after  each  model  selection  and  execution  to  ultimately  choose  the  "most-appropriate”  model 
considering  all  the  objectives  of  the  modeler. 
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CHAPTER  10 

SOFTWARE  RELIABILITY  EXECUTION  TIME  DATA  MODULE 


10.1  INTRODUCTION 

This  module  explains  the  execution  of  the  six  software  reliability  models  that  obtain  reliability 
estimates  and  predictions  for  execution  time  data.  Entry  of  a  Main  module  option  number  of  eight 
automatically  directs  die  program  to  this  module  if  TBF  data  are  resident  (i.e.,  the  data  type  was  entered 
as  a  one,  two,  or  three).  Chapter  1 1  explains  the  execution  of  the  five  software  reliability  models  that 
obtain  reliability  estimates  and  predictions  for  interval  data  (i.e.,  the  data  type  was  entered  as  a  four). 


10.2  EXECUTION  FLOW 

This  section  explains  the  prompts  and  inputs  for  die  software  reliability  portion  of  SMERFS 
pertaining  to  TBF  data.  The  placement  of  and  die  print  in  the  figures  conform  to  die  strategy  introduced 
in  Sections  1.5  and  1.6. 


a.  Time  Unit  Acceptability 

SMERFS  indicates  die  units  of  measurement  for  the  TBF  data.  If  WC  TBF  and  CPU  TBF 
data  are  resident  (data  type  was  entered  as  a  three),  die  units  of  both  are  listed.  The  user 
is  then  given  die  option  to  abort  the  analysis  if  the  units  are  not  acceptable  for  die  model 
estimations  and  predictions  (Figure  10-1).  For  example,  if  die  WC  TBF  data  are  stored  in 
seconds  and  the  predictions  are  desired  in  hours,  a  one  should  be  entered  to  abort  the 
analysis  and  return  control  to  the  Main  Module  Menu  (Chapter  2,  step  h).  The  data  should 
then  be  converted  to  hours  using  the  Unit  Conversions  module  (Chapter  5). 


THIS  DATA  ANALYSIS  WILL  BE  PERFORMED  IN  UNITS  CONSISTENT  WITH  THE 
STORED  DATA.  THIS  MEANS  THAT: 

TBF  ESTIMATIONS  AND  PREDICTIONS  WILL  BE  MADE  IN  atauai 

ENTER  ONE  TO  CONTINUE  WITH  THE  ANALYSIS;  ELSE  SERO  TO  RETURN  TO 
THE  MAIN  MODULE  MENU,  SO  CONVERSION  CAM  BE  MADE  BY  MODULE  THREE. 


FIGURE  10-1.  TIME  UNIT  ACCEPTABILITY  MESSAGE 
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b.  Software  Reliability  Model  Selection 

SMERFS  then  prompts  the  user  to  enter  the  desired  software  reliability  model  (Figure  10-2). 
If  an  invalid  option  number  is  entered,  the  same  list  and  second  prompt  are  output.  The  user 
response  to  the  third  prompt  in  the  figure  (which  is  only  issued  if  the  data  type  was  entered 
as  a  three)  is  used  to  determine  if  the  model  execution  is  to  be  performed  on  WC  TBF  or 
CPU  TBF  data. 


BRDt  TINS  MODEL  OPTION,  OR  EERO  FOR  A  LIST. 

THE  AVAILABLE  NALL  CLOCK  OR  CPU  TIKE  MODELS  ARE: 

1  THE  GEOMETRIC  MODEL 

2  THE  JELINSKX /MORANDA  DE-EUTROPHX CATION  MODEL 

3  THE  LITTLBWOOD  AMD  VERRALL  BAYESIAN  MODEL 

4  THE  MUSA  BASIC  EXECUTION  TIME  MODEL 

5  THE  MUSA  LOG  POISSON  EXECUTION  TIME  MODEL 

6  THE  NHPP  MODEL  FOR  TIME-BETWEEN-FAILURE  OCC. 

7  RETURN  TO  THE  MAIN  PROGRAM 
ENTER  MODEL  OPTION. 

If  VC  TBF  mnd  CPU  TBF  data,  than: 

ENTER  ONE  FOR  WC  TBF  OR  TWO  FOR  CPU  TBF. 

••DATA  TYPE  ERROR;  TRY  AGAIN  (AFTER  THE  PROMPT). 

Snd  if 


FIGURE  10-2.  EXECUTION  TIME  DATA  MODEL  MENU 


c.  Transfer  to  the  Indicated  Model 

If  the  entered  model  option  number  is  seven,  control  returns  to  the  Main  Module  Menu 
(Chapter  2,  step  h).  Otherwise  (if  a  valid  model  number  is  entered),  the  program  transfers 
control  to  the  indicated  model,  where: 

1.  The  Geometric  model  is  contained  in  Paragraph  10.2.1. 

2.  The  Jelinski/Moranda  model  is  contained  in  Paragraph  10.2.2. 

3.  The  Littlewood  and  Verrall  model  is  contained  in  Paragraph  10.2.3. 

4.  The  Musa  Basic  Execution  Time  model  is  contained  in  Paragraph  10.2.4. 

5.  The  Musa  Logarithmic  Poisson  Execution  Time  model  is  contained  in  Paragraph  10.2.3. 

6.  The  Non-homogeneous  Poisson  model  is  contained  in  Paragraph  10.2.6. 
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2.1  Geometric  Model 

If  the  entered  model  option  number  is  one,  then: 


a.  SMERFS  prompts  for  a  flag  indicating  whether  the  model  assumptions  and  requirements 
are  desired  (Figure  10-3).  If  desired,  the  listing  and  extra  prompt  in  the  figure  are  output. 
If  the  response  to  the  last  prompt  indicates  a  desire  to  abort  the  model  execution,  control 
returns  to  Section  10.2,  step  b.  (Refer  to  Paragraph  10.3.1  for  details  on  the  generated 
output.) 


mSI  On  FOR  GEOMETRIC  MODEL  DESCRIPTION;  ELSE  ZERO. 

THE  GEOMETRIC  MODEL  ASSUMPTIONS 

1.  THE  SOFTWARE  IS  OPERATED  IN  A  SIMILAR  MANNER  AS  THE  ANTICI¬ 
PATED  OPERATIONAL  USAGE. 

2.  ALL  FAULTS  ARE  EQUALLY  LIKELY  TO  OCCUR  AND  ARE  INDEPENDENT 
OF  EACH  OTHER. 

3.  THE  FAULT  DETECTION  RATE  FORMS  A  GEOMETRIC  PROGRESSION  AND 
IS  CONSTANT  BETWEEN  FAULT  OCCURRENCES. 


4.  THERE  IS  AN  INFINITE  NUMBER  OF  TOTAL  FAULTS  (I.E. ,  THE  PRO¬ 
GRAM  MILL  NEVER  BE  FAULT-FREE). 

MODEL  DATA  REQUIREMENTS 

1.  THE  TIME-BETWEEN-FAILURES  AS  MEASURED  IN  HALL  CLOCK  OR  COM¬ 
PUTER  CPU  TIME.  THE  DATA  SHOULD  HAVE  BEEN  ENTERED  VIA  THE 
INPUT  MODULE. 

IF  THE  ASSUMPTIONS  ARE  GENERALLY  SATISFIED  AND  THE  APPROPRIATE 
DATA  ARE  AVAILABLE,  ENTER  ONE  TO  CONTINUE  THE  MODEL;  OTHERWISE 
HERO  TO  RETURN  TO  THE  MENU  OF  MODELS. 


FIGURE  10-3.  MDLGEO  DESCRIPTION  PROMPTS  AND  LIST 


b.  SMERFS  prompts  the  user  to  enter  the  desired  method  of  estimation  [Maximum  Likelihood 
(ML)  or  Least  Squares  (LS)]  or  a  value  to  terminate  this  model  execution  (Figure  10-4). 
If  the  termination  value  is  entered,  control  branches  to  step  f. 


ENTER  ONE  FOR  MAXIMUM  LIF  HOOD  METHOD,  TWO  FOR  LEAST  SQUARES 
METHOD,  OR  THREE  TO  TERMI.wB  MODEL  EXECUTION. 


FIGURE  10-4.  MDLGEO  INPUT  PROMPT 
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c.  If  the  data  are  not  appropriate  for  this  model,  then  the  message  shown  in  Figure  10-S 
(including  the  ML  or  LS  indicator)  is  issued  and  control  returns  to  step  b. 


**THB  DATA  ARE  NOT  APPROPRIATE  FOR  THE  •*  MODEL. 


FIGURE  10-5.  MDLGEO  PROCESSING  ERROR  MESSAGE 


d.  Otherwise  Of  the  solution  is  obtained),  the  ML  or  LS  estimates  are  output  (Figure  10-6). 
Within  that  figure,  the  upper  report  (containing  the  95-percent  confidence  intervals) 
corresponds  to  the  ML  estimates  and  the  lower  report  to  the  LS  estimates. 


It  tli*  ML  Mt hod  was  salactad ,  than: 

ML  MODEL  ESTIMATES  ARE: 

(THE  APPROXIMATE  95%  CONFIDENCE  INTERVALS  APPEAR  IN  PARENTHESES) 


PROPORTIONALITY  CONSTANT  aaaaaaaaaa) 

HAZARD  RATE  PARAMETER  HUliNHa  (  hwhwm,  aaaaaaaaaa) 

INIT.  INTENSITY  FUNCTION  eeaaaaeeema 

am.  INTENSITY  FUNCTION  aaaaaaaaaaa 

MTBNF  HeeestMM  (  — — — — ,  M8mh«m) 

PURIFICATION  LEVEL  aematMM  (  eooeoee— o,  aaaaaaaaaa) 


Blse,  If  t ha  LS  method  was  selected,  them 


LS  MODEL  ESTIMATES  ARE: 

PROPORTIONALITY  CONSTANT  MMMeeeM 
HAZARD  RATE  PARAMETER  eeeeeeeeeae 
INIT.  INTENSITY  FUNCTION  aaeaaeeaaea 
CUR.  INTENSITY  FUNCTION  aaaaaaaaaaa 
MTBNF  aaaaaaaaaaa 
PURIFICATION  LEVEL  aeeaeeaaaee 


Bnd  it 


FIGURE  10-6.  MDLGEO  SUCCESSFUL  CONVERGENCE  OUTPUT 


e.  SMERFS  then  allows  for  the  processing  of  future  predictions  to  determine  the  expected 
performance  of  the  software  (Figure  10-7).  Upon  completion,  control  returns  to  step  b. 

ERROR:  If  the  estimate  for  the  MTBNF  is  negative,  these  future  predictions  are  not 
allowed  and  the  message  shown  at  the  bottom  of  the  figure  is  issued. 
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TBS  AVAILABLE  FUTURE  PREDICTIONS  ARE: 

1)  THE  TOTAL  MTBNF  FOR  THE  NEXT  K  FAILURES 

2)  THE  TIME  AND  NUMBER  OF  FAILURES  TO  REACH  A  DESIRED  INTENSITY 
FUNCTION 

3)  THE  NUMBER  OF  FAILURES  EXPECTED  IN  A  SPECIFIED  TIME 
ENTER  PREDICTION  OPTION,  OR  ZERO  TO  END  PREDICTIONS. 

ENTER  VALUE  OF  K,  OR  ZERO  TO  END. 

MTBNF  EXPECTED  •*•••«•••*« 

ENTER  INTENSITY  FUMC.  (LESS  THAN  •••••••••••) ,  OR  ZERO  TO  END. 

ADDITIONAL  TESTING  TIME  eetMtMHt 

#  OF  FAILURES  EXPECTED  tmeieMM 

ENTER  LENGTH  OF  THE  PERIOD,  OR  ZERO  TO  END. 

#  OF  FAILURES  EXPECTED  MmetMee 

** FUTURE  PREDICTIONS  ARE  NOT  ALLOWED  BECAUSE  THE  ESTIMATE  FOR  THE 
MTBNF  IS  LESS  THAN  ZERO. 


FIGURE  10-7.  MDLGEO  FUTURE  PREDICTION  PROMPTS  AND  OUTPUT 


f.  The  user  has  responded  to  the  prompt  in  step  b  with  a  value  equated  to  termination.  Before 
the  return  is  performed,  SMERFS  determines  if  the  last  execution  for  either  of  the  methods 
of  estimation  was  successful;  if  either  was,  it  allows  for  various  analyses  of  the  model  fit. 
This  is  shown  in  the  first  prompt  of  Figure  10-8.  Additionally,  if  both  the  ML  and  LS 
methods  were  successful,  the  second  prompt  is  issued,  allowing  the  user  to  select  the  set  of 
estimates  to  be  used  in  die  analyses.  Control  then  automatically  transfers  to  the  Analyses 
of  Model  Fit  sub-module  (Chapter  12).  Upon  completion,  control  returns  to  the  Execution 
Time  Data  Model  Menu  (Section  10.2,  step  b). 


ENTER  ONE  TO  PERFORM  AN  ANALYSIS  OF  THE  MODEL  FIT  USING  THE  PRE¬ 
DICTIONS  OF  THIS  MODEL;  ELSE  ZERO. 

If  ML  and  LS  vere  successful,  than: 

ENTER  ONE  FOR  PREDICTIONS  BASED  ON  MAXIMUM  LIKELIHOOD  METHOD,  OR 
TWO  FOR  LEAST  SQUARES  METHOD. 

Bad  if 


FIGURE  10-8.  MDLGEO  ANALYSES  OF  MODEL  FIT  PROMPTS 
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10.2.2  Jelinski/Moranda  De-Eutrophication  Model 


If  the  entered  model  option  number  is  two,  then: 

a.  SMERFS  prompts  for  a  flag  indicating  whether  the  model  assumptions  and  data  requirements 
are  desired  (Figure  10-9).  If  desired,  the  listing  and  extra  prompts  in  die  figure  are  output. 
If  die  response  to  die  last  prompt  indicates  a  desire  to  abort  die  model  execution,  control 
returns  to  Section  10.2,  step  b.  (Refer  to  Paragraph  10.3.2  for  details  on  the  generated 
output.) 


PITER  OMR  FOR  JELINSKI/MORANDA  MODEL  DESCRIPTION;  ELSE  XERO. 

THE  JELINSKI/MORANDA  “DE-EUTROPHICATION  MODEL*  ASSUMPTIONS 

1.  THE  SOFTWARE  IS  OPERATED  IN  A  SIMILAR  MANNER  AS  THE  ANTICI¬ 
PATED  OPERATIONAL  USAGE. 

2.  ALL  FAULTS  ARE  EQUALLY  LIKELY  TO  OCCUR  AND  ARE  INDEPENDENT 
OF  EACH  OTHER. 

3.  EACH  FAULT  IS  OF  THE  SAME  ORDER  OF  “IMPACT”  AS  ANY  OTHER. 

4.  THE  RATE  OF  FAULT  DETECTION  IS  PROPORTIONAL  TO  THE  CURRENT 
FAULT  CONTENT  OF  A  PROGRAM. 

5.  THE  FAILURE  RATE  REMAINS  CONSTANT  OVER  THE  TIME  PERIOD  BE¬ 
TWEEN  FAILURE  OCCURRENCES. 

6.  THE  FAULTS  ARE  CORRECTED  AT  THE  TIME  OF  OCCURRENCE  WITHOUT 
INTRODUCTION  OF  NEW  ONES. 

ENTER  ONE  TO  CONTINUE  THE  DESCRIPTION  LIST. 

MODEL  DATA  REQUIREMENTS 

li  THE  TIME-BETWEEN-FAILURES  AS  MEASURED  IN  WALL  CLOCK  OR  COM¬ 
PUTER  CPU  TIME.  THE  DATA  SHOULD  HAVE  BEEN  ENTERED  VIA  THE 
INPUT  MODULE. 

IP  THE  ASSUMPTIONS  ARE  GENERALLY  SATISFIED  AMD  THE  APPROPRIATE 

DATA  ARE  AVAILABLE,  ENTER  ONE  TO  CONTINUE  THE  MODEL;  OTHERWISE 

ZERO  TO  RETURN  TO  THE  MENU  OF  MODELS. 


FIGURE  10-9.  MDLJAM  DESCRIPTION  PROMPTS  AND  LIST 


b.  SMERFS  then  prompts  the  user  to  enter  the  desired  method  of  estimation  (ML  or  LS)  or  a 
value  to  terminate  this  model  execution  (Figure  10-10).  If  the  termination  value  is  entered, 
control  branches  to  step  f. 
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ENTER  am  FOR  MAXIMUM  LIKELIHOOD  METHOD,  TWO  FOR  LEAST  SQUARES 
METHOD,  OR  CTREE  TO  TERMINATE  MODEL  EXECUTION . 


FIGURE  10-10.  MDLJAM  INPUT  PROMPT 


c.  If  the  data  are  not  appropriate  for  this  model  or  the  estimate  of  the  total  number  of  faults  is 
less  than  the  observed  number,  then  the  appropriate  message  shown  in  Figure  10-11 
(including  the  ML  or  IS  indicator)  is  issued  and  control  returns  to  step  b. 


..  -  . 

♦♦THE  DATA  ARE  NOT  APPROPRIATE  FOR  THE  *1  MODEL. 

♦•THE  ESTIMATE  FOR  THE  TOTAL  NUMBER  OF  FAULTS  TO  BE  DETECTED  IN 
THE  PROGRAM  BEFORE  ALL  FAULTS  ARB  UNCOVERED  CAME  OUT  LESS  THAN 
THE  NUMBER  OF  FAULTS  FOUND  TO  DATE  WITHIN  THE  a*  MODEL. 


FIGURE  10-11.  MDLJAM  PROCESSING  ERROR  MESSAGES 


Otherwise  Of  the  solution  is  obtained),  the  ML  or  LS  estimates  are  output  (Figi  ~e  10-12). 


aa  MODEL  ESTIMATES  ARE: 
PROPORTIONALITY  CONSTANT 
INXT#  INTENSITY  FUNCTION 
CUR.  INTENSITY  FUNCTION 

JgepQJf  P  ■ 

TOTAL  NUMBER  OF  FAULTS 
#  OF  FAULTS  REMAINING 
PURIFICATION  LEVEL 


•aeaeeaeaaa 

eeaeaeasm 


aaaaaaaaaat 


FIGURE  10-12.  MDLJAM  SUCCESSFUL  CONVERGENCE  OUTPUT 


e.  SMERFS  then  allows  for  the  processing  of  future  predictions  to  determine  the  expected 
performance  of  the  software  (Figure  10-13).  Upon  completion,  control  returns  to  step  b. 

ERROR:  If  the  estimate  for  the  (rounded)  number  of  faults  remaining  is  less  than  one  or 
the  MTBNF  is  negative,  these  future  predictions  are  not  allowed  and  the 
appropriate  message  shown  at  the  bottom  of  the  figure  is  issued. 
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THE  AVAILABLE  FUTURE  PREDICTIONS  ARE: 

1)  THE  EXPECTED  RELIABILITY  FOR  A  SPECIFIED  TIME 

2)  THE  TOTAL  MTBNF  FOR  THE  NEXT  K  FAILURES 

3}  THE  TIME  AND  NUMBER  OF  FAILURES  TO  REACH  A  DESIRED  INTENSITY 
FUNCTION 

4)  THE  NUMBER  OF  FAILURES  EXPECTED  IN  A  SPECIFIED  TIME 
ENTER  PREDICTION  OPTION,  OR  ZERO  TO  END  PREDICTIONS. 

ENTER  LENGTH  OP  THE  PERIOD,  OR  ZERO  TO  END. 

FUTURE  RELIABILITY  eeeeaeaaeee 

ENTER  VALUE  OF  X,  OR  ZERO  TO  END. 

MTBNF  EXPECTED  eeeeeeeeeee 

ENTER  INTENSITY  FUNC.  (LESS  THAN  HIHIUIU),  OR  ZERO  TO  END. 
ADDITIONAL  TESTING  TIME  aeeeeaaaaae 

#  OF  FAILURES  EXPECTED  eeeeeeeaeee 

ENTER  LENGTH  OF  THE  PERIOD,  OR  ZERO  TO  END. 

#  OF  FAILURES  EXPECTED  eeeeeeeeeee 

** FUTURE  PREDICTIONS  ARE  NOT  ALLOWED  BECAUSE  THE  ESTIMATE  FOR  THE 
(ROUNDED)  NUMBER  OF  FAULTS  REMAINING  IS  LESS  THAN  ONE. 

** FUTURE  PREDICTIONS  ARE  NOT  ALLOWED  BECAUSE  THE  ESTIMATE  FOR  THE 
MTBNF  IS  LESS  THAN  ZERO. 


FIGURE  10-13.  MDLJAM  FUTURE  PREDICTION  PROMPTS  AND  OUTPUT 


f.  The  user  has  responded  to  the  prompt  in  step  b  with  a  value  equated  to  termination.  Before 
the  return  is  performed,  SMERFS  determines  if  the  last  execution  for  either  of  the  methods 
of  estimation  was  successful;  if  either  was,  it  allows  for  various  analyses  of  the  model  fit. 
This  is  shown  in  the  first  prompt  of  Figure  10-14.  Additionally,  if  both  the  ML  and  LS 
methods  were  successful,  die  second  prompt  is  issued,  allowing  the  user  to  select  the  set  of 
estimates  to  be  used  in  the  analyses.  Control  then  automatically  transfers  to  the  Analyses 
of  Model  Fit  sub-module  (Chapter  12).  Upon  completion,  control  returns  to  the  Execution 
Time  Data  Model  Menu  (Section  10.2,  step  b). 


ENTER  ONE  TO  PERFORM  AN  ANALYSIS  OF  THE  MODEL  FIT  USING  THE  PRE¬ 
DICTIONS  OF  THIS  MODEL;  ELSE  ZFRO. 

If  ML  and  LS  were  successful,  then: 

ENTER  ONE  FOR  PREDICTIONS  BASED  ON  MAXIMUM  LIKELIHOOD  METHOD,  OR 
TWO  FOR  LEAST  SQUARES  METHOD. 

End  if 


FIGURE  10-14.  MDLJAM  ANALYSES  OF  MODEL  FIT  PROMPTS 
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10.2.3  Littlewood  and  Verrall’s  Bavesian  Reliability  Growth  Model 


If  the  entered  model  option  number  is  three,  then: 


a.  SMERFS  prompts  for  a  flag  indicating  whether  the  model  assumptions  and  data  requirements 
are  desired  (Figure  10-15).  If  desired,  the  listing  and  extra  prompts  in  the  figure  are  output. 
If  the  response  to  the  last  prompt  indicates  a  desire  to  abort  the  model  execution,  control 
returns  to  Section  10.2,  step  b.  (Refer  to  Paragraph  10.3.3  for  details  on  the  prompted  input 
and  generated  output.) 


mn  ONE  FOR  LITTLEWOOD  &  VERRALL  MODEL  DESCR1PTXOM;  ELSE  EERO. 

THE  LITTLEWOOD  6  VERRALL  BAYESIAN  MODEL  ASSUMPTIONS 

1.  THE  SOFTWARE  IS  OPERATED  IN  A  SIMILAR  MANNER  AS  THE  ANTICI¬ 
PATED  OPERATIONAL  USAGE. 

2.  SUCCESSIVE  TXME-8ETWEEN-FAXLURES  ARE  INDEPENDENT  RANDOM 
VARIABLES  EACH  HAVING  AN  EXPONENTIAL  DISTRIBUTION.  THE  DIS¬ 
TRIBUTION  FOR  THE  I-TH  FAILURE  HAS  A  MEAN  OF  1/LAMDA(I). 

3.  THE  LAMDA(X)  FORM  A  SEQUENCE  OF  INDEPENDENT  VARIABLES,  EACH 
HAVING  A  GAMMA  DISTRIBUTION  WITH  THE  PARAMETERS  ALPHA  AND 
PHI (I).  PHI (I)  HAS  EITHER  THE  FORM: 

BETA ( 0 }  +  BETA ( 1 )  *  I  (LINEAR) 

OR 

BETA(O)  +  BETA(l)  *  1**2  (QUADRATIC). 

ENTER  ONE  TO  CONTINUE  THE  DESCRIPTION  LIST. 

MODEL  DATA  REQUIREMENTS 

1.  THE  TIME-BETWEEN-FAILURES  AS  MEASURED  IN  WALL  CLOCK  OR  COM¬ 
PUTER  CPU  TIME.  THE  DATA  SHOULD  HAVE  BEEN  ENTERED  VIA  THE 
INPUT  MODULE. 

IF  SHE  ASSUMPTIONS  ARE  GENERALLY  SATISFIED  AND  THE  APPROPRIATE 

DATA  ARE  AVAILABLE,  ENTER  ONE  TO  CONTINUE  THE  MODEL;  OTHERWISE 

ZERO  TO  RETURN  TO  THE  MENU  OF  MODELS. 


FIGURE  10-15.  MDLLAV  DESCRIPTION  PROMPTS  AND  LIST 


b.  SMERFS  then  prompts  the  user  to  enter  the  desired  method  of  estimation  (ML  or  LS)  or  a 
value  to  terminate  this  model  execution  (Figure  10-16).  If  the  termination  value  is  entered, 
control  branches  to  step  f.  If  a  method  of  estimation  is  selected,  then  SMERFS  prompts  for 
the  desired  function  number  (indicating  either  linear  or  quadratic).  For  the  LS  method,  no 
additional  inputs  are  required.  For  the  ML  method,  the  initial  estimates  for  BETA  are 
obtained  from  the  LS  method.  Those  values  are  then  output  (Figure  10-16)  and  SMERFS 
allows  the  user  to  override  the  values  to  ensure  the  global  maximum  has  been  readied. 
Lastly,  the  program  prompts  for  the  maximum  number  of  convergence  iterations. 
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ENTER  ONE  FOR  MAXIMUM  LIKELIHOOD  METHOD,  TWO  FOR  LEAST  8QUARES 
METHOD,  OR  THREE  TO  TERMINATE  MODEL  EXECUTION. 

WHICH  FUNCTION  IS  DESIRED  TO  BE  USED  AS  THE  PHI (I)  IN  THE  GAMMA 
DISTRIBUTION?  GAMMA  IS  USED  AS  THE  PRIOR  WITH  PARAMETERS  ALPHA 
AND  PHI (I) 

1.  PHI (X)  *  BETA(O)  +  BETA(l)  *  X  (LINEAR) 

OR 

2.  PHX(X)  *  BETA(O)  +  BETA(l)  *  1**2  (QUADRATIC). 

If  th e  ML  method  was  selected ,  than : 

THE  INITIAL  ESTIMATES  TO  BE  USED  IN  THE  ESTIMATION  PROCESS  ARE: 
BETA(O)  eeeeeeeeeee 

BETA ( 1 )  eeeeeeeeeee 

ENTER  ONE  TO  USE  DIFFERENT  INITIAL  ESTIMATES,  OR  SERO  TO  USE  THE 
INITIAL  ESTIMATES. 

If  user  override  was  selected,  then: 

ENTER  INITIAL  ESTIMATES  FOR  BETA(O)  AND  BETA(l) 

(BETWEEN  0 . 10000E-06  AND  0 . 10000E+08) . 

End  If 

ENTER  MAXIMUM  NUMBER  OF  ITERATIONS. 

End  If 


FIGURE  10-16.  MDLLAV  INPUT  PROMPTS 


c.  If  the  maximum  number  of  iterations  is  reached  before  a  solution  is  found,  the  Trust  Region 
could  not  be  adjusted  properly  using  the  initial  estimates  for  BETA,  or  the  model  parameters 
resulted  in  a  negative  initial  MTBNF,  then  the  appropriate  message  shown  in  Figure  10-17 
(including  the  ML  or  LS  indicator)  is  issued  and  control  returns  to  step  b. 


**THE  MAXIMUM  NUMBER  OF  ITERATIONS  WAS  REACHED  WITH  THE  FOLLOWING 

FINAL  ML  MODEL  ESTIMATES: 

ALPHA 

eeeeeeeeeee 

8ETA(0) 

eeeeeeeeeee 

BETA(l) 

FUNCTION  EVALUATED  AT 

eeeeeeeeeee 

THESE  POINTS 

eeeeeeeeeee 

INIT.  INTENSITY  FUNCTION 

eeeeeeeeeee 

CUR.  INTENSITY  FUNCTION 

eeeeeeeeeee 

MTBNF 

eeeeeeeeeee 

FIGURE  10-17.  MDLLAV  PROCESSING  ERROR  MESSAGES 


10-10 


NSWCDD  TR  84-373 


••THE  TRUST  REGION  COULD  NOT  BE  ADJUSTED  PROPERLY  IN  TOE  ML  MODEL. 

••THE  ESTIMATES  OF  THE  aa  MODEL  PARAMETERS: 

BETA(O)  eeeeeeeeeee  AND  BETA(l)  eeeeeeeeeee 
RESULTED  IN  A  NEGATIVE  ESTIMATE  FOR  THE  INITIAL  MTBNF.  THE  MODEL 
THUS  CANNOT  BE  APPLIED  FOR  THIS  DATA  SET. 


FIGURE  10-17.  MDLLAV  PROCESSING  ERROR  MESSAGES  (Continued) 


d.  Otherwise  (if  the  solution  is  obtained),  the  ML  or  LS  estimates  are  output  (Figure  10-18). 
Within  the  figure,  the  upper  report  (containing  the  additional  ALPHA  estimate  and  die 
number  of  iterations  performed  to  obtain  the  estimates)  corresponds  to  the  ML  estimates  and 
the  lower  report  to  the  LS  estimates. 


1  If  the  ML  method  was  selected,  then: 

I  ML  MODEL  ESTIMATES  AFTER  iii  ITERATIONS  ARE: 

ALPHA 

eeeeeeeeeee 

BETA(O) 

eeeeeeeeeee 

BETA  { 1 ) 

FUNCTION  EVALUATED  AT 

eeeeeeeeeee 

j  THESE  POINTS 

eeeeeeeeeee 

INIT.  INTENSITY  FUNCTION 

eeeeeeeeeee 

CUR.  INTENSITY  FUNCTION 

eeeeeeeeeee 

MTBNF 

. 

eeeeeeeeeee 

|  Else,  if  the  LS  method  was  selected,  then: 

j  LS  MODEL  ESTIMATES  ARE:  1 

BETA(O) 

eeeeeeeeeee  . 

BETA(l) 

FUNCTION  EVALUATED  AT 

eeeeeeeeeee 

< 

THESE  POINTS 

eeeeeeeeeee 

INIT.  INTENSITY  FUNCTION 

eeeeeeeeeee 

CUR.  INTENSITY  FUNCTION 

eeeeeeeeeee 

MTBNF 

End  if 

eeeeeeeeeee 

FIGURE  10-18.  MDLLAV  SUCCESSFUL  CONVERGENCE  OUTPUT 


e.  SMERFS  then  allows  for  the  processing  of  future  predictions  to  determine  the  expected 
performance  of  the  software  (Figure  10-19).  Within  the  figure,  the  upper  report  (containing 
the  prediction  for  the  additional  testing  time  and  the  expected  number  of  failures  to  reach 
a  desired  intensity  function)  corresponds  to  the  linear  function  and  the  lower  report  to  the 
quadratic  function.  Upon  completion,  control  returns  to  step  b. 
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ERROR:  If  the  MTBNF  is  negative,  these  future  predictions  are  not  allowed  and  the 
message  shown  at  the  bottom  of  the  figure  is  issued. 


If  the  Linear  /unction  was  selected,  thmnt 

THE  AVAILABLE  FUTURE  PREDICTIONS  ARE: 

1)  THE  TOTAL  MTBNF  FOR  THE  NEXT  X  FAILURES 

2)  THE  TIME  AND  NUMBER  OF  FAILURES  TO  REACH  A  DESIRED  INTENSITY 
FUNCTION 

3)  THE  NUMBER  OF  FAILURES  EXPECTED  IN  A  SPECIFIED  TIME 
ENTER  PREDICTION  OPTION,  OR  ZERO  TO  END  PREDICTIONS. 

ENTER  VALUE  OF  X,  OR  ZERO  TO  END. 

MTBNF  EXPECTED 

ENTER  INTENSITY  FUNC.  (LESS  THAN  •■•••••••■•),  OR  ZERO  TO  END. 

ADDITIONAL  TESTING  TIME  eeeeeeeeeee 
4  OF  FAILURES  EXPECTED  eeeeeeaaaae 

ENTER  LENGTH  OF  THE  PERIOD,  OR  ZERO  TO  END. 

#  OF  FAILURES  EXPECTED  eeeeeeeeeee 

Elm*,  if  the  Quadratic  function  warn  selected,  them 

THE  AVAILABLE  FUTURE  PREDICTIONS  ARE: 

1)  THE  TOTAL  MTBNF  FOR  THE  NEXT  K  FAILURES 

2)  THE  NUMBER  OF  FAILURES  EXPECTED  IN  A  SPECIFIED  TIME 
ENTER  PREDICTION  OPTION,  OR  ZERO  TO  END  PREDICTIONS. 

ENTER  VALUE  OF  K,  OR  ZERO  TO  END. 

MTBNF  EXPECTED  eeeeeeeeeee 

ENTER  LENGTH  OF  THE  PERIOD,  OR  ZERO  TO  END. 

4  OF  FAILURES  EXPECTED  eeeeeeeeeee 

End  df 

♦•FUTURE  PREDICTIONS  ARE  NOT  ALLOWED  BECAUSE  THE  ESTIMATE  FOR  THE 
MTBNF  IS  LESS  THAN  ZERO. 


FIGURE  10-19.  MDLLAV  FUTURE  PREDICTION  PROMPTS  AND  OUTPUT 


f.  The  user  has  responded  to  the  prompt  in  step  b  with  a  value  equated  to  termination.  Before 
the  return  is  performed,  SMERFS  determines  if  the  last  execution  for  either  of  the  methods 
of  estimation  was  successful;  if  either  was,  it  allows  for  various  analyses  of  the  model  fit. 
This  is  shown  in  the  first  prompt  of  Figure  10-20.  Additionally,  if  both  the  ML  and  LS 
methods  were  successful,  die  second  prompt  is  issued,  allowing  die  user  to  select  the  set  of 
estimates  to  be  used  in  the  analyses.  Control  then  automatically  transfers  to  the  Analyses 
of  Model  Fit  sub-module  (Chapter  12).  Upon  completion,  control  returns  to  the  Execution 
Time  Data  Model  Menu  (Section  10.2,  step  b). 
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INTER  ONE  TO  PERFORM  AN  ANALYSIS  OF  TEE  MODEL  FIT  USING  TEE  PRE¬ 
DICTIONS  OF  THIS  MODEL;  ELSE  ZERO. 

It  HL  and  LS  vara  successful,  than: 

ENTER  ONE  FOR  PREDICTIONS  RASED  ON  MAXIMUM  LIKELIHOOD  METHOD,  OR 
TWO  FOR  LEAST  SQUARES  METHOD. 

End  it 

FIGURE  10-20.  MDLLAV  ANALYSES  OF  MODEL  FIT  PROMPTS 
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10.2.4  Musa’s  Execution  Time  Model 

If  the  entered  model  option  number  is  four,  then: 

a.  SMERFS  prompts  for  a  flag  indicating  whether  the  model  assumptions  and  data  requirements 
are  desired  (Figure  10-21).  If  desired,  the  listing  and  extra  prompts  in  the  figure  are  output. 
If  the  response  to  the  last  prompt  indicates  a  desire  to  abort  the  model  execution,  control 
returns  to  Section  10.2,  step  b.  (Refer  to  Paragraph  10.3.4  for  details  on  the  prompted  input 
and  generated  output.) 


Smut  on  FOR  MUSA  (BASIC)  MODEL  DESCRIPTION;  ELSE  ZERO. 

THE  MUSA  BASIC  EXECUTION  TIME  MODEL  ASSUMPTIONS 

1.  THE  SOFTWARE  IS  OPERATED  IN  A  SIMILAR  MANNER  AS  THE  ANTICI¬ 
PATED  OPERATIONAL  USAGE. 

2.  ALL  FAULTS  ARE  EQUALLY  LIKELY  TO  OCCUR  AND  ARE  INDEPENDENT 
OF  EACH  OTHER. 

3.  ALL  SOFTWARE  FAULTS  ARE  OBSERVED. 

4.  THE  EXECUTION  TIMES  (MEASURED  IN  CPU  TIME)  BETWEEN  FAILURES 
ARE  PIECEWISE  EXPONENTIALLY  DISTRIBUTED. 

5.  THE  INTENSITY  FUNCTION  IS  PROPORTIONAL  TO  THE  NUMBER  OF 
FAULTS  REMAINING  IN  THE  PROGRAM. 

6.  THE  FAULT  CORRECTION  RATE  IS  PROPORTIONAL  TO  THE  FAILURE 
OCCURRENCE  RATE. 

ENTER  ONE  TO  CONTINUE  THE  DESCRIPTION  LIST. 

MODEL  DATA  REQUIREMENTS 

I.  THE  TIME-BETWEEN-FA1LURES  AS  MEASURED  IN  COMPUTER  CPU  TIME. 
THE  DATA  SHOULD  HAVE  BEEN  ENTERED  VIA  THE  INPUT  MODULE. 

IF  THE  ASSUMPTIONS  ARE  GENERALLY  SATISFIED  AND  THE  APPROPRIATE 
DATA  ARE  AVAILABLE,  ENTER  ONE  TO  CONTINUE  THE  MODEL;  OTHERWISE 
EERO  TO  RETURN  TO  THE  MENU  OF  MODELS. 


FIGURE  10-21.  MDLMUS  DESCRIPTION  PROMPTS  AND  LIST 


b.  If  the  data  are  not  appropriate  for  this  model  or  the  estimate  of  the  total  number  of  faults  is 
less  than  the  observed  number,  then  the  appropriate  message  shown  in  Figure  10-22  is  issued 
and  control  branches  to  step  g. 


10-14 


NSWCDD  TR  84-373 


FIGURE  10-24.  MDLMUS  FUTURE  PREDICTION  PROMPTS  AND  OUTPUT 
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II OKX  VALUE  or  X  (BETWEEN  ONE  AND  •••••••••••),  ok  zsko  to  end. 

MTBNF  EXPECTED  ••naatMM 

INTER  INTENSITY  TONC.  (LESS  THAN  OR  EERO  TO  END. 

ADDITIONAL  TESTING  TIME  MMaeamM 

#  OP  FAILURES  EXPECTED 

ENTER  DESIRED  NTBNF  (GREATER  THAN  •••••••••••),  OR  EERO  TO  END. 

ADDITIONAL  TESTING  TIME 

#  OP  FAILURES  EXPECTED  ••••••«»•• 

ENTER  LBNOTB  OF  THE  PERIOD,  OR  EBRO  TO  END. 

#  OF  FAILURES  EXPECTED  •—•••—* 

♦•FUTURE  PREDICTIONS  ARE  NOT  ALLOWED  BECAUSE  TUB  ESTIMATE  FOR  THE 
(ROUNDED)  NUMBER  OF  FAULTS  REMAINING  IS  LESS  THAN  ONE. 

••FUTURE  PREDICTIONS  ARB  NOT  ALLOWED  BECAUSE  THE  ESTIMATE  FOR  THE 
MTBNF  IS  LESS  THAN  ZERO. 


FIGURE  10-24.  MDLMUS  FUTURE  PREDICTION  PROMPTS  AND  OUTPUT  (Continued) 


e.  SMERFS  then  prompts  for  a  flag  indicating  whether  the  Calendar  Time  Component  of  the 
main  model  is  desired  (Figure  10*25).  If  desired,  a  second  prompt  is  issued  to  determine 
if  the  assumptions  and  data  requirements  are  desired.  If  desired,  die  listing  and  extra 
prompts  in  the  figure  are  output.  If  the  response  to  the  last  prompt  indicates  a  desire  to 
abort  the  component  execution,  control  branches  to  step  g. 


ENTER  ONE  TO  WORK  WITH  CALENDAR  TIME  COMPONENT;  ELSE  ZERO. 

ENTER  ONE  FOR  CALENDAR  TIME  COMPONENT  DESCRIPTION;  ELSE  ZERO. 
CALENDAR  TIME  MODEL  ASSUMPTIONS 

1.  THE  QUANTITIES  OF  THE  AVAILABLE  RESOURCES  ( FAILURE-IDENTI¬ 
FICATION  PERSONNEL,  FAILURE-CORRECTION  PERSONNEL,  AND  COM¬ 
PUTER  TIME)  ARE  CONSTANT  OVER  TESTING  SEGMENTS. 

2.  RESOURCE  EXPENDITURES  FOR  THE  ITH  RESOURCE  ASSOCIATED  WITH 
A  CHANGE  IN  MTBNF  FROM  T1  TO  T2  CAN  BE  APPROXIMATED  BY 

THETA(I)  *  DELTA (TAU)  +  MU(I)  *  DELTA (M) 

WHERE 

THETA ( I )  -  EXECUTION  TIME  COEFFICIENT  OF  RESOURCE  EX¬ 

PENDITURE  FOR  THE  ITH  RESOURCE 


FIGURE  10-25.  MDLMUS  CALENDAR  TIME  COMPONENT  DESCRIPTION 
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DELTA (TAU)-  INCREMENT  07  EXECUTION  TIME  ASSOCIATED  WITH 
THE  CHANGE  IN  MTBNF 

ENTER  ONE  TO  CONTINUE  THE  DESCRIPTION  LIST. 

MU(I)  -  FAILURE  COEFFICIENT  OF  RESOURCE  EXPENDITURE 
FOR  THE  ITH  RESOURCE 

DELTA (M)  -  THE  INCREMENT  OF  FAILURES  EXPERIENCED  WITH 

THE  CHANGE  IN  MTBNF 

3.  FAILURE  IDENTIFICATION  PERSONNEL  CAN  BE  FULLY  UTILIZED  AND 
COMPUTER  UTILIZATION  IS  CONSTANT. 

4.  FAILURE-CORRECTION  PERSONNEL  UTILIZATION  IS  ESTABLISHED  BY 
LIMITATION  OF  ERROR  QUEUE  LENGTH  FOR  ANY  DEBUGGER.  ERROR 
QUEUE  LENGTH  IS  DETERMINED,  IN  TURN,  BY  ASSUMING  THE  ERROR 
CORRECTION  PROCESS  IS  POISSON  AND  THAT  THE  FAILURE-CORREC¬ 
TION  PERSONNEL  ARE  RANDOMLY  ASSIGNED  ERRORS  OVER  TIME. 

ENTER  ONE  TO  CONTINUE  THE  DESCRIPTION  LIST. 

MODEL  DATA  REQUIREMENTS 

1.  THE  AVAILABLE  RESOURCES  FOR  BOTH  FAILURE-IDENTIFICATION  AND 
FAILURE-CORRECTION  PERSONNEL. 

2.  THE  NUMBER  OF  COMPUTER  SHIFTS. 


3. 


THE  EXECUTION  TIME  COEFFICIENT  OF  RESOURCE  EXPENDITURE  FOR 
EACH  RESOURCE. 


4.  THE  FAILURE  COEFFICIENT  OF  RESOURCE  EXPENDITURE  FOR  EACH 
RESOURCE. 

5.  THE  MAXIMUM  ERROR  QUEUE  LENGTH  FOR  EACH  DEBUGGER. 

ENTER  ONE  TO  CONTINUE  THE  DESCRIPTION  LIST. 

6.  THE  PROBABILITY  THAT  THE  ERROR  QUEUE  LENGTH  IS  NO  LARGER 
THAN  THE  MAXIMUM  QUEUE  LENGTH. 

7.  THE  COMPUTER  UTILIZATION  RATIO. 

8.  THE  DESIRED  MTBNF. 

IF  THE  ASSUMPTIONS  ARE  GENERALLY  SATISFIED  AND  THE  APPROPRIATE 
DATA  ARE  AVAILABLE,  ENTER  ONE  TO  CONTINUE  THE  MODEL;  OTHERWISE 
HERO  TO  SKIP  THE  CALENDAR  TINE  MODEL. 


FIGURE  10-25.  MDLMUS  CALENDAR  TIME  COMPONENT  DESCRIPTION  (Continued) 


f.  SMERFS  then  prompts  and  inputs  the  12  values  needed  to  perform  the  Calendar  Time 
Component,  as  shown  in  Figure  10-26.  (Note  that  after  the  additional  testing  time  prediction 
is  output,  SMERFS  allows  the  user  to  repeat  the  execution  with  only  the  desired  MTBNF 
changed  or  additionally  with  all  12  values  reprompted.) 
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ENTER  THE  APPROPRIATE  DATA  AFTER  EACH  PROMPT 

1  HUMBER  OP  FAILURE  IDENTIFICATION  PERSONNEL. 

2  NUMBER  OF  FAILURE  CORRECTION  PERSONNEL. 

3  NUMBER  OF  COMPUTER  SHIFTS  (EXAMPLE  -  IF  THE  MORE  WEEK  IS  40 
HRS.  AND  THE  COMPUTER  IS  AVAILABLE  SO  HRS.  A  WEEK,  THEM  EN¬ 
TER  THE  MUMBER  2) . 

4  AVERAGE  AMOUNT  OF  COMPUTER  (WALLCLOCX)  TIME  EXPENDED  PER 
UNIT  OF  EXECUTION  TIME  (CPU). 

5  AVERAGE  AMOUNT  OF  IDENTIFICATION  WORK  TIME  EXPENDED  PER 
UNIT  OF  EXECUTION  TIME. 

6  AVERAGE  AMOUNT  OF  COMPUTER  TIME  (HRS)  EXPENDED  PER  FAILURE. 

7  AVERAGE  AMOUNT  OF  IDENTIFICATION  WORK  (HRS)  REQUIRED  PER 
FAILURE. 

8  AVERAGE  AMOUNT  OF  CORRECTION  WORK  (HRS)  REQUIRED  PER  FAIL¬ 
URE. 

9  MAXIMUM  ERROR  QUEUE  LENGTH  FOR  A  DEBUGGER. 

10  ASSOCIATED  PROBABILITY  THAT  THE  QUEUE  LENGTH  WILL  BE  MO 
LARGER  THAN  VALUE  JUST  INPUT. 

11  COMPUTER  UTILIZATION  FACTOR  (0.LT.X.LE.1) . 

12  ENTER  A  DESIRED  MTBJfF  FOR  THE  PROGRAM  (IN  HOURS  AMD  LARGER 
THAN  THE  CURRENT  ESTIMATED  VALUE  OF  ■••••••••••). 

ADDITIONAL  HOURS  TO  MTBNF  eeeeeeeeeee 

ENTER  ONE  TO  USE  A  DIFFERENT  SPECIFIED  MTBNF;  ELSE  ZERO. 

ENTER  ONE  FOR  ALL  NEW  CALENDAR  TIME  MODEL  INPUTS;  ELSE  ZERO. 


FIGURE  10-26.  MDLMUS  COMPONENT  PROMPTS  AND  OUTPUT 


g.  SMERFS  determines  if  the  execution  was  successful.  If  it  was,  it  allows  for  various 
analyses  of  the  model  fit.  This  is  shown  in  the  prompt  of  Figure  10-27.  Control  then 
automatically  transfers  to  the  Analyses  of  Model  Fit  sub-module  (Chapter  12).  Upon 
completion,  control  returns  to  the  Execution  Time  Data  Model  Menu  (Section  10.2,  step  b). 


ENTER  ONE  TO  PERFORM  AN  ANALYSIS  OF  THE  MODEL  FIT  USING  THE  PRE¬ 
DICTIONS  OF  THIS  MODEL;  ELSE  ZERO. 


□ 


FIGURE  10-27.  MDLMUS  ANALYSES  OF  MODEL  FIT  PROMPTS 
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10.2.5  Musa’s  Logarithmic  Poisson  Execution  Time  Model 
If  the  entered  model  option  number  is  five,  then: 

a.  SMERFS  prompts  for  a  flag  indicating  whether  the  model  assumptions  and  data  requirements 
are  desired  (Figure  10-28).  If  desired,  the  listing  and  extra  prompts  in  the  figure  are  output. 
If  the  response  to  the  last  prompt  indicates  a  desire  to  abort  the  model  execution,  control 
returns  to  Section  10.2,  step  b.  (Refer  to  Paragraph  10.3.5  for  details  on  die  prompted  input 
and  generated  output.) 


non  OMB  FOB  MUSA  (LOO)  MODEL  DESCRIPTION;  ELSE  SESO. 

THE  MUSA  LOGARITHMIC  POISSON  EXECUTION  TIME  MODEL  ASSUMPTIONS 

1.  THE  SOFTWARE  IS  OPERATED  IN  A  SIMILAR  MANNER  AS  THE  ANTICI¬ 
PATED  OPERATIONAL  USAGE. 

2.  ALL  FAULTS  ARE  EQUALLY  LIKELY  TO  OCCUR  AND  ARE  INDEPENDENT 
OF  EACH  OTHER. 

3.  THE  EXPECTED  NUMBER  OF  FAULTS  IS  A  LOGARITHMIC  FUNCTION  OF 
TIME. 

4.  THE  FAILURE  INTENSITY  DECREASES  EXPONENTIALLY  WITH  THE  EX¬ 
PECTED  FAILURES  EXPERIENCED. 

5.  THE  SOFTWARE  WILL  EXPERIENCE  AN  INFINITE  NUMBER  OF  FAIL¬ 
URES. 

ENTER  ONE  TO  CONTINUE  THE  DESCRIPTION  LIST. 

MODEL  DATA  REQUIREMENTS 

1.  THE  TIME^BETWEEN-FAILURES  AS  MEASURED  IN  COMPUTER  CPU  TIME. 
THE  DATA  SHOULD  HAVE  BEEN  ENTERED  VIA  THE  INPUT  MODULE. 

IF  TEE  ASSUMPTIONS  ARE  GENERALLY  SATISFIED  AND  THE  APPROPRIATE 
DATA  ARE  AVAILABLE,  ENTER  ONE  TO  CONTINUE  THE  MODEL;  OTHERWISE 
EERO  TO  RETURN  TO  THE  MENU  OF  MODELS. 


FIGURE  10-28.  MDLMSA  DESCRIPTION  PROMPTS  AND  LIST 


b.  If  the  data  are  not  appropriate  for  this  model,  then  the  message  shown  in  Figure  10-29  is 
issued  and  control  branches  to  step  e. 


FIGURE  10-29.  MDLMSA  PROCESSING  ERROR  MESSAGE 
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c.  Otherwise  (if  the  solution  is  obtained),  the  ML  estimates  are  output  (Figure  10-30). 


THE  MAXIMUM  LIKELIHOOD  ESTIMATES  ARE: 

BETAO  PARAMETER 

eeeeeeeeeee 

BETA1  PARAMETER 

eeeeeeeeeee 

INIT.  INTENSITY  FUNCTION 

eeeeeeeeeee 

CUR.  INTENSITY  FUNCTION 

eeeeeeeeeee 

CUR.  MTBNF 

eeeeeeeeeee 

PURIFICATION  LEVEL 

eeeeeeeeeee 

FIGURE  10-30.  MDLMSA  SUCCESSFUL  CONVERGENCE  OUTPUT 


d.  SMERFS  then  allows  for  the  processing  of  future  predictions  to  determine  die  expected 
performance  of  the  software  (Figure  10-31). 

ERROR:  If  the  estimate  for  the  (rounded)  number  of  faults  remaining  is  less  than  one  or 
the  MTBNF  is  negative,  these  future  predictions  are  not  allowed  and  die 
appropriate  message  shown  at  the  bottom  of  the  figure  is  issued. 


THE  AVAILABLE  FUTURE  PREDICTIONS  ARE: 

1)  THE  EXPECTED  RELIABILITY  FOR  A  SPECIFIED  TIME 

2)  THE  TOTAL  MTBNF  FOR  THE  NEXT  K  FAILURES 

3)  THE  TIME  AND  NUMBER  OF  FAILURES  TO  REACH  A  DESIRED  INTENSITY 
FUNCTION 

4)  THE  NUMBER  OF  FAILURES  EXPECTED  IN  A  SPECIFIED  TIME 
ENTER  PREDICTION  OPTION,  OR  ZERO  TO  END  PREDICTIONS. 

ENTER  LENGTH  OF  THE  PERIOD,  OR  ZERO  TO  END. 

FUTURE  RELIABILITY  eeeeeeeeeee 


ENTER  VALUE  OF  K,  OR  ZERO  TO  END. 

MTBNF  EXPECTED  eeeeeeeeeee 

ENTER  INTENSITY  FUNC.  (LESS  THAN  eeeeeeeeeee),  OR  ZERO  TO  END. 
ADDITIONAL  TESTING  TIME  eeeeeeeeeee 
#  OF  FAILURES  EXPECTED  eeeeeeeeeee 

ENTER  LENGTH  OF  THE  PERIOD,  OR  ZERO  TO  END. 

4  OF  FAILURES  EXPECTED  eeeeeeeeeee 

** FUTURE  PREDICTIONS  AND  PREDICTED  TBFS  ARE  NOT  ALLOWED  BECAUSE 
THE  BETA0  PARAMETER  OF  THE  MODEL  IS  LESS  THAN  OR  EQUAL  TO  ONE. 

** FUTURE  PREDICTIONS  ARE  NOT  ALLOWED  BECAUSE  THE  ESTIMATE  FOR  THE 
MTBNF  IS  LESS  THAN  ZERO. 


FIGURE  10-31.  MDLMSA  FUTURE  PREDICTION  PROMPTS  AND  OUTPUT 
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e.  SMERFS  determines  if  the  execution  was  successful  and  if  the  indicated  BETA  parameter 
is  greater  than  one.  If  both  tests  showed  true,  SMERFS  allows  for  various  analyses  of  the 
model  fit.  This  is  shown  in  the  prompt  of  Figure  10-32.  Control  then  automatically 
transfers  to  the  Analyses  of  Model  Fit  sub-module  (Chapter  12).  Upon  completion,  control 
returns  to  the  Execution  Time  Data  Model  Menu  (Section  10.2,  step  b). 


If  til*  parameter  BET  AO  allows  for  the  TBF  prediction ,  them 

EM TER  ONE  SO  PERFORM  AN  ANALYSIS  OF  THE  MODEL  FIT  USING  SHE  PRE¬ 
DICTIONS  OF  SHIS  MODEL;  ELSE  ZERO. 

End  it 


FIGURE  10-32.  MDLMSA  ANALYSES  OF  MODEL  FIT  PROMPTS 
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10.2.6  Non-homoyeneous  Poisson  for  Execution  Time  Data  Model 


If  the  entered  model  option  number  is  six,  then: 

a.  SMERFS  prompts  for  a  flag  indicating  whether  the  model  assumptions  and  data  requirements 
are  desired  (Figure  10-33).  If  desired,  the  listing  and  extra  prompts  in  the  figure  are  output. 
If  the  response  to  the  last  prompt  indicates  a  desire  to  abort  the  model  execution,  control 
returns  to  Section  10.2,  step  b.  (Refer  to  Paragraph  10.3.6  for  details  on  the  prompted  input 
and  generated  output.) 


ENTER  ONE  FOR  NHPP  MODEL  DESCRIPTION;  ELSE  ZERO. 

THE  NON-HOMOGENEOUS  POISSON  MODEL  ASSUMPTIONS 

1.  THE  SOFTWARE  IS  OPERATED  IN  A  SIMILAR  MANNER  AS  THE  ANTICI¬ 
PATED  OPERATIONAL  USAGE. 

2.  ALL  FAULTS  ARE  EQUALLY  LIKELY  TO  OCCUR  AND  ARE  INDEPENDENT 
OF  EACH  OTHER. 

3.  THE  CUMULATIVE  NUMBER  OF  FAULTS  DETECTED  AT  ANY  TIME  FOL¬ 
LOWS  A  POISSON  DISTRIBUTION  WITH  MEAN  M(T) .  THAT  MEAN  IS 
SUCH  THAT  THE  EXPECTED  NUMBER  OF  FAULTS  IN  ANY  SMALL  TIME 
INTERVAL  ABOUT  T  IS  PROPORTIONAL  TO  THE  NUMBER  OF  UNDETEC¬ 
TED  FAULTS  AT  TIME  T. 

4.  THE  MEAN  IS  ASSUMED  TO  BE  A  BOUNDED  NON-DECREASING  FUNCTION 
WITH  M(T)  APPROACHING  IN  THE  LIMIT,  "A"  (THE  EXPECTED  TOTAL 
NUMBER  OF  FAULTS  TO  BE,  EVENTUALLY,  DETECTED  IN  THE  TESTING 
PROCESS),  AS  THE  LENGTH  OF  TESTING  GOES  TO  INFINITY. 

ENTER  ONE  TO  CONTINUE  THE  DESCRIPTION  LIST. 

MODEL  DATA  REQUIREMENTS 

1.  THE  TIME-BETWEEN-FAILURES  AS  MEASURED  IN  WALL  CLOCK  OR  COM¬ 
PUTER  CPU  TIME.  THE  DATA  SHOULD  HAVE  BEEN  ENTERED  VIA  THE 
INPUT  MODULE. 

IF  THE  ASSUMPTIONS  ARE  GENERALLY  SATISFIED  AND  THE  APPROPRIATE 
DATA  ARE  AVAILABLE,  ENTER  ONE  TO  CONTINUE  THE  MODEL;  OTHERWISE 
ZERO  TO  RETURN  TO  THE  MENU  OF  MODELS. 


FIGURE  10-33.  MDLNPT  DESCRIPTION  PROMPTS  AND  LIST 


b.  If  the  data  are  not  appropriate  for  this  model  or  the  estimate  of  the  total  number  of  faults  is 
less  than  the  observed  number,  then  the  appropriate  message  shown  in  Figure  10*34  is  issued 
and  control  branches  to  step  e. 
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**THE  DATA  ARE  NOT  APPROPRIATE  FOR  THE  MODEL. 

**THE  ESTIMATE  FOR  THE  TOTAL  NUMBER  OF  FAULTS  TO  BE  DETECTED  IN 
THE  PROGRAM  BEFORE  ALL  FAULTS  ARE  UNCOVERED  CAME  OUT  LESS  THAN 
THE  NUMBER  OF  FAULTS  FOUND  TO  DATE  WITHIN  THE  MODEL. 


FIGURE  10-34.  MDLNPT  PROCESSING  ERROR  MESSAGES 


Otherwise  (if  the  solution  is  obtained),  the  ML  estimates  are  output  (Figure  10-35). 


MODEL  ESTIMATES  ARE: 

PROPORTIONALITY  CONSTANT 

eeeeeeeeeee 

INIT.  INTENSITY  FUNCTION 

eeeeeeeeeee 

CUR.  INTENSITY  FUNCTION 

eeeeeeeeeee 

TOTAL  NUMBER  OF  FAULTS 

eeeeeeeeeee 

#  OF  FAULTS  REMAINING 

eeeeeeeeeee 

PURIFICATION  LEVEL 

eeeeeeeeeee 

FIGURE  10-35.  MDLNPT  SUCCESSFUL  CONVERGENCE  OUTPUT 


SMERFS  then  allows  for  the  processing  of  future  predictions  to  determine  the  expected 
performance  of  the  software  (Figure  10-36). 

ERROR:  If  the  estimate  for  the  (rounded)  number  of  faults  remaining  is  less  than  one, 
these  future  predictions  are  not  allowed  and  the  message  shown  at  the  bottom  of 
the  figure  is  issued. 


THE  AVAILABLE  FUTURE  PREDICTIONS  ARE: 

1)  THE  EXPECTED  RELIABILITY  FOR  A  SPECIFIED  TIME 

2)  THE  TIME  TO  REACH  A  SPECIFIED  RELIABILITY  FOR  A  SPECIFIED 
OPERATIONAL  TIME 

3)  THE  TIME  TO  REACH  A  DESIRED  INTENSITY  FUNCTION 

4)  THE  NUMBER  OF  FAILURES  EXPECTED  IN  A  SPECIFIED  TIME 
ENTER  PREDICTION  OPTION,  OR  ZERO  TO  END  PREDICTIONS. 

ENTER  LENGTH  OF  THE  PERIOD,  OR  ZERO  TO  END. 

FUTURE  RELIABILITY  eeeeeeeeeee 

ENTER  DESIRED  RELIABILITY,  OR  ZERO  TO  END. 

ENTER  SPECIFIED  OPERATIONAL  TIME. 

ADDITIONAL  TESTING  TIME  eeeeeeeeeee 


FIGURE  10-36.  MDLNPT  FUTURE  PREDICTION  PROMPTS  AND  OUTPUT 
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ENTER  INTENSITY  PUNC.  (LESS  THAN  •••••••••••),  OR  ZERO  TO  END. 

ADDITIONAL  TESTING  TIME  eeeeeeeees* 

ENTER  LENGTH  OF  THE  PERIOD,  OR  ZERO  TO  END. 

#  OF  FAILURES  EXPECTED  eeeeeeeeese 

♦•FUTURE  PREDICTIONS  ARE  NOT  ALLOWED  BECAUSE  THE  ESTIMATE  FOR  THE 
(ROUNDED)  NUMBER  OF  FAULTS  REMAINING  IS  LESS  THAN  ONE. 


FIGURE  10-36.  MDLNPT  FUTURE  PREDICTION  PROMPTS  AND  OUTPUT  (Continued) 


e.  SMERFS  determines  if  the  execution  was  successful.  If  it  was,  it  allows  for  various 
analyses  of  the  model  fit.  This  is  shown  in  the  prompt  of  Figure  10-37.  Control  then 
automatically  transfers  to  the  Analyses  of  Model  Fit  sub-module  (Chapter  12).  Upon 
completion,  control  returns  to  the  Execution  Time  Data  Model  Menu  (Section  10.2,  step  b). 


ENTER  ONE  TO  PERFORM  AN  ANALYSIS  OF  THE  MODEL  FIT  USING  THE  PRE¬ 
DICTIONS  OF  THIS  MODEL;  ELSE  ZERO. 


FIGURE  10-37.  MDLNPT  ANALYSES  OF  MODEL  FIT  PROMPTS 
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10.3  MODEL  OUTPUTS  AND  CONSIDERATIONS 

The  reader  is  directed  to  the  references  for  a  detailed  description  of  the  model  parameters  and 
outputs  covered  in  this  chapter.  Reference  2  is  especially  applicable  unless  indicated  otherwise  in  this 
section.  This  section  briefly  discusses  the  meaning  of  required  inputs  and  displayed  outputs  for  each 
model.  Considerations  for  requested  inputs  are  also  discussed.  One  of  the  biggest  changes  from 
SMERFS  version  3  to  4  (thus  incorporated  in  version  S)  is  that  most  of  the  models  no  longer  require 
inputted  starting  values  for  the  optimization  procedures  for  model  parameter  estimation.  The  models 
using  the  Dekker-Brent  optimization  procedure  have  built-in  calculations  for  such  values  and,  as  a  result, 
no  user  input  is  required.  The  only  model  not  employing  this  technique  in  this  section  is  the  Littlewood 
and  Verrall  Bayesian  Reliability  Growth  Model.  This  model  employs  the  Trust  Region  methodology. 
This  technique  will  still,  however,  determine  starting  values  or  let  the  user  override  them  by  inputting 
different  values. 

The  time  uni's  for  all  outputs  of  a  selected  model  are  expressed  in  those  units  selected  by  the  user 
for  the  data  set.  This  is  true  for  all  models  except  the  Calendar  Time  Component  of  Musa’s  Basic 
Execution  Time  model.  For  that  component,  the  output  is  expressed  in  the  number  of  hours  required  to 
achieve  the  desired  reliability  level,  regardless  of  the  units  of  the  failure  data. 


10.3.1  Geometric  Model 

Besides  the  estimates  of  the  parameters  defining  the  model,  the  output  includes  estimates  for: 

a.  Initial  intensity  function 

b.  Current  intensity  function 

c.  MTBNF 

d.  Current  purification  level 

The  intensity  function  is  how  fast  the  expected  MTBNF  function  is  changing  with  respect  to  time.  The 
initial  is  at  time  t=0,  while  the  current  is  at  the  present  time.  The  purification  level  is  the  relative  "fault- 
freeness"  of  the  code.  The  closer  the  value  is  to  one,  the  "purer"  is  the  code.  When  all  faults  have  been 
found  and  eliminated,  the  value  is  exactly  one. 

For  prediction  purposes,  the  model  provides: 

a.  Total  MTBNF  for  the  next  K  failures 

b.  Time  and  number  of  failures  to  reach  a  desired  intensity  function 

c.  Number  of  failures  expected  in  a  specified  time 

The  total  MTBNF  for  the  next  K  failures  is  the  expected  length  of  time  (expressed  in  the  user-selected 
units)  until  exactly  K  failures  will  have  occurred.  The  time  and  number  of  failures  to  reach  a  desired 
intensity  function  will  provide  an  estimate  of  how  long  it  will  take  and  how  many  failures  are  to  be 
expected  until  a  desired  intensity  function  (failures  /  unit  of  time)  is  achieved.  The  last  prediction  statistic 
is  self-explanatory.  All  of  these  quantities  can  be  used  for  tradeoff  analyses  to  determine  optimal  release 
time  of  the  software  or  to  determine  when  a  given  operational  program  is  a  prime  candidate  for  a  rewrite. 
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10.3.2  Jelinski/Moranda  De-Eutrophication  Model 

Besides  the  same  outputs  provided  by  the  Geometric  model  (Paragraph  10.3.1),  the  Jelinski/ 
Moranda  model  also  gives  an  estimate  of  the  total  number  of  faults  and  the  number  of  faults  remaining 
in  the  code.  The  number  remaining  is  simply  obtained  from  the  estimate  of  the  total  number  of  faults 
by  subtracting  the  number  found  to  date. 

In  addition  to  the  same  prediction  estimates  as  provided  by  die  Geometric  model,  this  model  also 
allows  the  user  to  obtain  an  estimate  of  the  expected  reliability  (i.e.,  probability  of  the  software  not 
failing)  over  a  specified  time  period  that  the  user  enters.  Again,  tradeoff  analyses  can  be  used  to 
determine  optimal  release  or  rewrite  time. 


10.3.3  Littlewood  and  VerralPs  Bavesian  Reliability  Growth  Model 

If  the  user  decides  to  override  the  initial  estimates  for  BETA(0)  and  BETA(l),  (e.g.,  explore  for 
a  global  maximum),  the  values  that  are  input  should  be  in  the  range  1.0E-07  and  1.0E+07.  Values  for 
BETA(0),  in  addition,  should  be  tried  between  0  and  1  for  initial  exploration.  The  "FUNCTION"  that 
is  mentioned  in  the  output  (see  Figure  10-17)  is  either  the  likelihood  function  in  the  case  of  the  maximum 
likelihood  estimates  or  the  residual  sums-of-squares  for  the  least  squares  estimates  (Reference  2).  This 
quantity  is  especially  useful  in  determining  when  a  global  solution  has  been  obtained. 

The  model  output  includes  estimates  for: 

a.  Initial  intensity  function 

b.  Current  intensity  function 

c.  Current  MTBNF 

The  intensity  function  is  the  derivative  of  the  MTBNF  function  with  respect  to  time.  The  initial  is  this 
derivation  at  time  t=0,  and  the  current  is  the  derivation  at  the  present  time.  It  provides  the  user  an  idea 
of  the  rate  at  which  failures  are  occurring  at  a  given  point  in  time. 

For  prediction  purposes,  the  model  provides: 

a.  Total  MTBNF  for  the  next  K  failures 

b.  Time  and  number  of  failures  to  reach  a  desired  intensity  function 

c.  Number  of  failures  expected  in  a  specified  time 

The  total  MTBNF  for  the  next  K  failures  is  the  expected  length  of  time  (expressed  in  the  user-selected 
units)  until  exactly  K  failures  will  have  occurred.  The  time  and  number  of  failures  to  reach  a  desired 
intensity  function  (which  is  only  available  for  the  linear  option)  will  provide  an  estimate  of  how  long  it 
will  take  and  how  many  failures  are  to  be  expected  until  a  desired  intensity  function  (failures  /  unit  of 
time)  is  achieved.  The  last  prediction  statistic  is  self-explanatory.  All  of  these  quantities  can  be  used  for 
tradeoff  analyses  to  determine  optimal  release  time  of  the  software  or  to  determine  when  a  given 
operational  program  is  a  prime  candidate  for  a  rewrite. 
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10.3.4  Musa’s  Execution  Time  Model 

If  convergence  is  achieved,  the  SMERFS  program  estimates,  in  addition  to  the  model  parameters, 
include: 

a.  Initial  intensity  function 

b.  Current  intensity  function 

c.  Initial  MTBNF 

d.  Current  MTBNF 

d.  Total  number  of  faults 

e.  Number  of  faults  remaining 

f.  Purification  level 

The  initial  intensity  function  is  the  rate  at  which  failures  are  occurring  at  time  t=0,  and  die  current  is  the 
rate  at  die  present  time.  The  purification  level  is  the  relative  "fault-freeness"  of  the  code.  The  closer 
die  value  is  to  one,  the  "purer”  is  the  code.  When  all  faults  have  been  found  and  eliminated,  die  value 
is  exactly  one. 

Besides  all  of  the  prediction  statistics  that  the  Geometric  model  (Paragraph  10.3.1)  provides,  this 
model  also  allows  the  user  to  estimate  the  number  of  failures  required  to  reach  a  desired  MTBNF.  All 
of  these  prediction  statistics  can  be  used  for  tradeoff  analyses  to  determine  optimal  release  time  of  die 
software  or  to  determine  when  a  given  operational  program  is  a  prime  candidate  for  a  rewrite. 

For  the  input  to  the  Calendar  Time  Component,  most  of  the  prompts  are  self-explanatory.  The 
failure  identification  personnel  are  the  testers  and  die  failure  correction  personnel  are  the  program 
maintainers.  The  maximum  queue  length  for  a  debugger  is  the  maximum  number  of  errors  that  are 
assigned  to  any  one  programmer  to  correct.  Finally,  the  computer  utilization  factor  is  that  fraction  of 
capacity  at  which  the  computer  operates.  The  additional  hours  that  are  needed  to  meet  the  specified 
MTBNF  is  expressed  in  WC  hours  rather  than  CPU  time  shown  in  die  first  part  of  die  model  output. 

For  a  more  detailed  explanation  of  all  these  quantities,  see  Reference  5. 


10.3.5  Musa’s  Logarithmic  Poisson  Execution  Time  Model 

Besides  the  model  parameters,  the  output  includes  all  of  the  outputs  of  the  Musa  Basic  Execution 
Time  model  (Paragraph  10.3.4)  except  for  the  initial  MTBNF  and  the  total  number  of  faults  (therefore 
die  total  remaining),  since  this  model  assumes  that  an  infinite  number  of  faults  are  present  in  die  code. 

For  prediction  statistics,  all  have  been  considered  already  in  the  previous  description  for  the 
Geometric  model  (Paragraph  10.3.1)  and  the  Jelinski/Moranda  model  (Paragraph  10.3.2). 
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10.3.6  Non-homogenous  Poisson  Model  for  Execution  Time  Data 

Besides  the  estimates  of  the  model  parameters,  the  output  includes  estimates  for: 

a.  Initial  intensity  function 

b.  Current  intensity  function 

c.  Total  number  of  faults 

d.  Number  of  faults  remaining 

e.  Purification  level 

See  die  description  of  the  Musa  Basic  Execution  Tune  model  (Paragraph  10.3.4)  for  explanations  of  these 
quantities. 

For  prediction  statistics,  the  outputs  are: 

a.  Expected  reliability  for  a  specified  time 

b.  Time  to  reach  a  specified  reliability  for  a  specified  operational  time 

c.  Time  to  reach  a  desired  intensity  function 

d.  Number  of  failures  expected  in  a  specified  time 

For  the  expected  reliability  (i.e.,  probability  of  die  software  not  failing)  for  a  specified  time,  the  user 
inputs  a  given  length  in  die  user-specified  units  of  time  and  die  program  will  provide  an  estimate  of  that 
reliability.  The  program  can  provide  the  user  an  estimate  of  die  required  time  needed  to  achieve  a 
specified  reliability  for  a  specified  operational  time.  For  example,  if  .93  was  entered  for  die  desired 
reliability  with  a  specified  operational  time  (measured  in  CPU  time)  of  10  hours,  the  resulting  output 
would  be  the  amount  of  testing  time  (again  measured  in  CPU  time)  that  would  have  to  be  expended  to 
achieve  the  stated  goal.  If  the  user  were  to  expend  this  amount,  the  probability  would  be  .95  that  the 
software  would  operate  10  CPU  hours  without  failure.  The  remaining  prediction  statistics  have  been 
considered  previously  (Paragraph  10.3.1). 


10-28 


NSWCDD  TR  84-373 


CHAPTER  11 

i 

SOFTWARE  RELIABILITY  INTERVAL  DATA  MODULE 


11.1  INTRODUCTION 

This  module  explains  the  execution  of  five  software  reliability  models  that  obtain  reliability 
estimates  and  predications  for  interval  data.  Entry  of  a  Main  module  option  number  of  eight 
automatically  directs  foe  program  to  this  module  if  interval  data  are  resident  (i.e.,  die  data  type  was 
entered  as  a  four).  Chapter  10  explains  the  execution  of  the  six  software  reliability  models  that  obtain 
reliability  estimates  and  predictions  for  TBF  data  (i.e.,  the  data  type  was  entered  as  a  one,  two,  or  three). 


11.2  EXECUTION  FLOW 

This  section  explains  the  prompts  and  inputs  for  the  software  reliability  portion  of  SMERFS 
pertaining  to  interval  data.  The  placement  of  and  die  print  in  die  figures  conform  to  die  strategy 
introduced  in  Sections  1.5  and  1.6. 

a.  Software  Reliability  Model  Selection 

SMERFS  first  prompts  the  user  to  enter  the  desired  software  reliability  model  (Figure  11-1). 
If  an  invalid  option  number  is  entered,  the  same  list  and  second  prompt  are  output. 


8MXBR  COUNT  MODEL  OPTION,  OR  XERO  FOR  A  LIST. 
THE  AVAILABLE  FAULT  COUNT  MODELS  ARE t 

1  THE  BROOKS  AND  MOTLEY  MODEL 

2  THE  GENERALIZED  POISSON  MODEL 

3  THE  NON— HOMOGENEOUS  POISSON  MODEL 

4  THE  i :  SCHNEXDEHIND  MODEL 

5  THE  S-SHAPED  RELIABILITY  GROWTH  MODEL 

6  RETURN  TO  THE  MAIN  PROGRAM 
ENTER  MODEL  OPTION. 


FIGURE  11-1.  INTERVAL  DATA  MODEL  MENU 
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b.  Transfer  to  the  Indicated  Model 

If  the  entered  model  option  number  is  six,  control  returns  to  the  Main  Module  Menu 
(Chapter  2,  step  h).  Otherwise  (if  a  valid  model  number  is  entered),  the  program  transfers 
control  to  the  indicated  model,  where: 

1.  The  Brooks  and  Motley  model  is  contained  in  Paragraph  11.2.1. 

2.  The  Generalized  Poisson  model  is  contained  in  Paragraph  11.2.2. 

3.  The  Non-homogen eous  Poisson  model  is  contained  in  Paragraph  11.2.3. 

4.  The  Schneidewind  model  is  contained  in  Paragraph  11.2.4. 

3.  The  S-Shaped  Reliability  Growth  model  is  contained  in  Paragraph  11.2.3. 
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11.2.1.  Brooks  and  Motley’s  Discrete  Software  Reliability  Model 


If  the  entered  model  option  number  is  one,  then: 


a.  SMERFS  prompts  for  a  flag  indicating  whether  the  model  assumptions  and  data  requirements 
are  desired  (Figure  1 1-2).  If  desired,  the  listing  and  extra  prompts  in  the  figure  are  output. 
If  the  response  to  the  last  prompt  indicates  a  desire  to  abort  the  model  execution,  control 
returns  to  Section  1 1 .2,  step  a.  (Refer  to  Paragraph  1 1 .3. 1  for  details  on  the  prompted  input 
and  generated  output.) 


ENTER  OKI  FOR  BROOKS  AND  MOTLEY  MODEL  DESCRIPTION;  ELSE  ZERO. 

THE  BROOKS  AND  MOTLEY  DISCRETE  SR  MODEL  ASSUMPTIONS 

1.  THE  SOFTWARE  IS  OPERATED  IN  A  SIMILAR  MANNER  AS  THE  ANTICI¬ 
PATED  OPERATIONAL  USAGE. 

2.  ALL  FAULTS  ARE  EQUALLY  LIKELY  TO  OCCUR  AND  ARE  INDEPENDENT 
OF  EACH  OTHER. 

3.  EACH  FAULT  IS  OF  THE  SAME  ORDER  OF  "IMPACT"  AS  ANY  OTHER. 

4.  THE  NUMBER  OF  SOFTWARE  FAULTS  DETECTED  ON  EACH  TESTING  OC¬ 
CASION  IS  PROPORTIONAL  TO  THE  NUMBER  OF  FAULTS  AT  RISK  FOR 
DETECTION.  THIS,  IN  TURN,  IS  PROPORTIONAL  TO  THE  NUMBER  OF 
FAULTS  REMAINING. 

5.  THE  PROPORTIONALITY  FACTOR  OR  PROBABILITY  (DENOTED  AS  Q  FOR 
THE  BINOMIAL  MODEL  AND  PHI  FOR  THE  POISSON  MODEL)  OF  DETEC¬ 
TING  ANY  FAULT  DURING  A  SPECIFIED  UNIT  INTERVAL  OF  TESTING 
IS  CONSTANT  OVER  ALL  OCCASIONS  AND  INDEPENDENT  OF  THE  FAULT 
DETECTIONS. 

■  ENTER  ONE  TO  CONTINUE  THE  DESCRIPTION  LIST. 

6.  THE  FAULTS  REINTRODUCED  IN  THE  CORRECTION  PROCESS  ARE  PRO¬ 
PORTIONAL  TO  THE  NUMBER  OF  FAULTS  DETECTED. 

MODEL  DATA  REQUIREMENTS 

1.  THE  NUMBER  OF  FAULTS  DETECTED  IN  EACH  INTERVAL  OF  TESTING. 

2.  THE  LENGTHS  OF  THE  VARIOUS  TESTING  INTERVALS. 

"162  SHOULD  HAVE  BEEN  ENTERED  VIA  THE  DATA  INPUT  MODULE. " 

IF  THE  ASSUMPTIONS  ARE  GENERALLY  SATISFIED  AND  THE  APPROPRIATE 
DATA  ARE  AVAILABLE,  ENTER  ONE  TO  CONTINUE  THE  MODEL;  OTHERWISE 
EERO  TO  RETURN  TO  THE  MENU  OF  MODELS. 


FIGURE  11-2.  MDLBAM  DESCRIPTION  PROMPTS  AND  LIST 


11-3 


NSWCDD  TR  84-373 

b.  SMERFS  then  permits  an  extended  description  of  the  Binomial  and  Poisson  models,  by 
issuing  the  prompt  shown  in  Figure  11-3.  If  the  response  entered  indicates  that  the 
additional  information  is  desired,  the  listing  in  the  figure  is  output. 


ENTER  ONE  FOR  DETAILS  ON  THE  BINOMIAL  AMD  POISSON  METHODS;  ELSE 
SERO. 

IN  THE  BINOMIAL  MODEL,  IT  IS  ASSUMED  THAT  THE  NUMBER  OF  FAULTS 
DETECTED  IN  THE  I-TH  INTERVAL  OF  TESTING  FOLLOWS  A  BINOMIAL 
DISTRIBUTION  WITH  PARAMETERS  N(I)  AND  Q(I),  WHERE: 

Q(l)  -  1  -  (1  -  Q)  **  T(I) ; 

T(I)  IS  THE  LENGTH  OF  THE  I-TH  TESTING  PERIOD,  AND 
Q  IS  THE  FAULTS  DETECTION  PROBABILITY, 

AND 

N(I)  -  N  *  W(I)  -  ALPHA  *  NCUM(I-l)  ; 

N  IS  THE  TOTAL  NUMBER  OF  FAULTS  IN  THE  PROGRAM, 

W(X)  IS  THE  FRACTION  OF  THE  PROGRAM  BEING  TESTED  ON  THE 
I-TH  TESTING  OCCASION, 

NCUM(I-l)  IS  THE  CUMULATIVE  NUMBER  OF  FAULTS  FOUND  IN  THIS 
SECTION  OF  THE  CODE  THROUGH  THE  (I-l)ST  TESTING  PERIOD  AND 
ALPHA  IS  THE  PROBABILITY  OF  CORRECTING  FAULTS  WITHOUT  RE¬ 
INSERTING  NEW  ONES. 

ENTER  ONE  TO  CONTINUE  THE  DESCRIPTION  LIST. 

IN  THE  POISSON  MODEL,  IT  IS  ASSUMED  THAT  THE  NUMBER  OF  FAULTS 
DETECTED  IN  THE  I-TH  INTERVAL  OF  TESTING  FOLLOWS  A  POISSON 
DISTRIBUTION  WITH  PARAMETER 
LAMDA(l)  «  N(I)  *  PHI ( I }  WHERE: 

PHI (I)  «  1  -  (1  -  PHI)  **  T(I) ; 

T(I)  IS  THE  LENGTH  OF  THE  I-TH  TESTING  PERIOD,  AND 
PHI  IS  THE  FAULT  DETECTION  PROBABILITY 

AND 

N<I)  -  N  *  W(I)  -  ALPHA  *  NCUM(I-l) ; 

N  IS  THE  TOTAL  NUMBER  OF  FAULTS  IN  THE  PROGRAM, 

W(I)  IS  THE  FRACTION  OF  THE  PROGRAM  BEING  TESTED  ON  THE 
I-TH  TESTING  OCCASION, 

NCUM(I-l)  IS  THE  CUMULATIVE  NUMBER  OF  FAULTS  FOUND  IN  THIS 
SECTION  OF  THE  CODE  THROUGH  THE  (X-X)ST  TESTING  PERIOD,  « 
ALPHA  IS  THE  PROBABILITY  OF  CORRECTING  FAULTS  WITHOUT  RE¬ 
INSERTING  NEW  ONES. 


FIGURE  11-3.  MDLBAM  EXTENDED  DESCRIPTION  PROMPTS  AND  UST 


c.  SMERFS  prompts  for  a  flag  indicating  whether  the  program  testing  plan  involved  the  entire 
program  or  only  a  fraction  of  the  program,  as  shown  in  the  first  prompt  of  Figure  1 1-4.  If 
the  testing  included  the  entire  program,  SMERFS  constructs  both  die  fractional  portion 
vector  and  the  vector  containing  the  number  of  faults  detected  (in  that  fractional  portion)  in 
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the  previous  testing  periods.  Otherwise,  SMERFS  prompts  the  user  to  enter  both  values  for 
each  testing  period,  as  shown  in  the  second  prompt  of  the  figure. 


ENTER  ONE  TO  INPUT  THE  FRACTION  OF  THE  PROGRAM  BEING  TESTED  AT 
EACH  TESTING  PERIOD,  OR  ZERO  TO  SET  THE  FRACTIONS  ALL  EQUAL  TO  I 
(I.E.,  THE  ENTIRE  PROGRAM  WAS  UNDER  TEST  IN  EACH  OCCASION). 

If  a  positive  response  was  input ,  then  s  series  of: 

ENTER  FRACTION  OF  CODE  BEING  TESTED  IN  PERIOD  ill  AND  THE  NUM¬ 
BER  OF  FAULTS  DETECTED  IN  THE  FRACTIONAL  PART  IN  THE  PREVIOUS 
ii  TEST  PERIODS. 

End  if 


FIGURE  11-4.  MDLBAM  FRACTION  OF  CODE  UNDER  TEST  PROMPTS 


d.  SMERFS  prompts  the  user  to  enter  the  desired  model  (Binomial  or  Poisson)  or  a  value  to 
terminate  this  model  execution  (Figure  1 1-5).  If  the  termination  value  is  entered,  control 
branches  to  step  h.  If  a  desired  model  is  selected,  then  the  initial  estimates  for  both  the 
probability  of  detecting  faults  and  the  total  number  of  faults  are  internally  established. 
Those  values  are  then  output  (Figure  1 1-5)  and  SMERFS  allows  the  user  to  override  the 
values  to  ensure  that  the  global  maximum  has  been  reached.  The  program  prompts  for  the 
probability  that  a  fault  will  not  be  inserted  while  one  is  being  corrected.  Lastly,  the  program 
prompts  for  the  maximum  number  of  convergence  iterations. 


r  ENTER  ONE  FOR  THE  BINOMIAL  MODEL,  TWO  FOR  THE  POISSON  MODEL,  OR 
THREE  TO  TERMINATE  MODEL  EXECUTION. 

THE  INITIAL  ESTIMATES  TO  BE  USED  IN  THE  ESTIMATION  PROCESS  ARE: 
PROB.  OF  DETECTING  FAULTS  eeeeeeeeeee 
TOTAL  NUMBER  OF  FAULTS  eeeeeeeeeee 

I  ENTER  ONE  TO  USE  DIFFERENT  INITIAL  ESTIMATES,  OR  ZERO  TO  USE  THE 
INITIAL  ESTIMATES. 

Jf  user  override  was  selected,  then: 

ENTER  INITIAL  ESTIMATE  FOR  THE  PROBABILITY  OF  DETECTING  FAULTS 
(BETWEEN  0 . 10000E-06  AND  0 .99999E-00) . 

ENTER  INITIAL  ESTIMATE  FOR  THE  TOTAL  NUMBER  OF  FAULTS 
(BETWEEN  eeeeeeeeeee  AND  0 . 10000E+08) . 

End  if 


FIGURE  11-5.  MDLBAM  INPUT  PROMPTS 
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ENTER  SHE  DESIRED  ALPHA  (PROBABILITY  OF  CORRECTING  FAULTS  IM  THE 
PROGRAM  WITHOUT  INSERTING  NEW  (WES) . 

ENTER  MAXIMUM  NUMBER  OP  ITERATIONS. 


FIGURE  11-5.  MDLBAM  INPUT  PROMPTS  (Continued) 


e.  If  the  maximum  number  of  iterations  is  reached  before  a  solution  is  found,  the  Trust  Region 
could  not  be  adjusted  properly  using  the  values  for  the  initial  estimates,  or  the  estimate  of 
the  total  number  of  faults  is  Iks  than  the  observed  number,  then  the  appropriate  message 
shown  in  Figure  11-6  (including  the  BINOMIAL  or  POISSON  indicator)  is  issued  and 
control  returns  to  step  d. 


**THE  MAXIMUM  NUMBER  OF  ITERATIONS  WAS  REACHED  WITH  THE  FOLLOWING 
FINAL  mau&a  MODEL  ESTIMATES: 

PROB.  OF  DETECTING  FAULTS  n««ee«MM 
TOTAL  NUMBER  OF  FAULTS  aeNMaMM 
4  OF  FAULTS  REMAINING  aaeeaeeaaaa 

••THE  TRUST  REGION  COULD  NOT  BE  ADJUSTED  PROPERLY  FOR  THE  UU1UI 
MODEL. 

••THE  ESTIMATE  FOR  THE  TOTAL  NUMBER  OF  FAULTS  TO  BE  DETECTED  IN 
THE  PROGRAM  BEFORE  ALL  FAULTS  ARE  UNCOVERED  CAME  OUT  LESS  THAN 
THE  NUMBER  OF  FAULTS  FOUND  TO  DATE  WITHIN  THE  aaaaaaaa  MODEL. 


FIGURE  11-6.  MDLBAM  PROCESSING  ERROR  MESSAGES 


f.  Otherwise  (if  the  solution  is  obtained),  the  BINOMIAL  or  POISSON  estimates  are  output, 
including  the  number  of  iterations  performed  to  obtain  those  estimates  (Figure  11-7). 


THE  aaaaaaaa  MODEL  ESTIMATES,  AFTER  ill  ITERATIONS  ARE: 
PROB.  OF  DETECTING  FAULTS  eeeeeeeeeee 
TOTAL  NUMBER  OF  FAULTS  eeeeeeaeeee 

4  OF  FAULTS  REMAINING  eeeeeeeeeea 


FIGURE  11-7.  MDLBAM  SUCCESSFUL  CONVERGENCE  OUTPUT 


g.  SMERFS  then  allows  for  the  processing  of  future  predictions  to  determine  the  expected 
performance  of  the  software  (Figure  1 1-8).  Upon  completion  control  returns  to  step  d. 
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NOTE  1 :  The  SMERFS  program  attempts  to  minimize  the  number  of  prompted  values  by 
cycling  only  on  the  final  value  (i.e.,  the  values  entered  for  the  fraction  of  the 
program  being  tested  and  the  number  of  faults  discovered  in  that  section  are 
prompted  only  once;  the  length  is  prompted  until  the  zero  is  entered).  If 
changing  of  one  of  the  "fixed'  values  is  desired,  a  zero  should  be  entered  to  end 
the  current  predictions,  and  then  the  same  prediction  option  re-elected.  All  the 
values  will  then  be  prompted. 

NOTE  2:  Certain  predictions  can  result  in  infinite  looping;  the  SMERFS  program  guards 
against  this  condition  through  an  internal  program  maximum.  If  that  maximum 
is  reached,  an  error  message  is  issued.  Additionally,  the  user  is  allowed  to 
change  that  maximum  if  it  is  felt  that  the  maximum  is  too  high  or  too  low  (refer 
to  the  second  predictions  of  the  figure). 

ERROR:  If  the  estimate  for  the  (rounded)  number  of  faults  remaining  is  less  than  one, 
these  future  predictions  are  not  allowed  and  the  message  shown  at  the  bottom  of 
the  figure  is  issued. 


THE  AVAILABLE  FUTURE  PREDICTIONS  ARE: 

1)  THE  NUMBER  OF  FAULTS  EXPECTED  IN  THE  NEXT  TESTING  PERIOD 

2)  THE  NUMBER  OF  PERIODS  NEEDED  TO  DISCOVER  THE  NEXT  M  FAULTS 
ENTER  PREDICTION  OPTION,  OR  ZERO  TO  END  PREDICTIONS. 

ENTER  FRACTION  OF  THE  PROGRAM  TO  BE  TESTED  (FOR  ENTIRE  PROGRAM, 
ENTER  ONE). 

BOW  MANY  FAULTS  HAVE  BEEN  FOUND  TO  DATE  IN  THE  SECTION  OF  THE 
CODE  TO  BE  TESTED? 

ENTER  PROJECTED  LENGTH  OF  THE  PERIOD,  OR  ZERO  TO  END. 

#  OF  FAULTS  EXPECTED  eeeeeeeeeee 

ENTER  PROJECTED  LENGTH  OF  THE  PERIODS. 

ENTER  FRACTION  OF  THE  PROGRAM  TO  BE  TESTED  (FOR  ENTIRE  PROGRAM, 
ENTER  ONE) . 

HOW  MANY  FAULTS  HAVE  BEEN  FOUND  TO  DATE  IN  THE  SECTION  OF  THE 
CODE  TO  BE  TESTED? 

ENTER  VALUE  OF  M  (BETWEEN  ONE  AND  eeeeeeeeeee),  OR  ZERO  TO  END. 

#  OF  PERIODS  EXPECTED  eeeeeeeeeee 

If  the  maximum  iterations  (1000)  is  reached,  then: 

**#  OF  PERIODS  EXCEEDS  eeeeeeeeeee  (PROGRAM  MAXIMUM) 

ENTER  ONE  TO  CHANGE  THE  PROGRAM  MAXIMUM;  ELSE  ZERO. 

ENTER  THE  NEW  VALUE  POR  THE  PROGRAM  MAXIMUM. 

End  if 

** FUTURE  PREDICTIONS  ARE  NOT  ALLOWED  BECAUSE  THE  ESTIMATE  FOR  THE 
(ROUNDED)  NUMBER  OF  FAULTS  REMAINING  IS  LESS  THAN  ONE. 


FIGURE  11-8.  MDLBAM  FUTURE  PREDICTION  PROMPTS  AND  OUTPUT 
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h.  The  user  has  responded  to  the  prompt  in  step  d  with  a  value  equated  to  termination.  Before 
the  return  is  performed,  SMERFS  determines  if  the  last  execution  for  either  of  the  methods 
of  estimation  was  successful;  if  either  was,  it  allows  for  various  analyses  of  the  model  fit. 
This  is  shown  in  the  first  prompt  of  Figure  11-9.  Additionally,  if  both  the  Binomial  and 
Poisson  methods  were  successful,  the  second  prompt  is  issued,  allowing  the  user  to  select 
the  set  of  estimates  to  be  used  in  the  analyses.  Control  then  automatically  transfers  to  the 
Analyses  of  Model  Fit  sub-module  (Chapter  12).  Upon  completion,  control  returns  to  the 
Interval  Data  Model  Menu  (Section  11.2,  step  a). 


ENTER  ONE  TO  PERFORM  AN  ANALYSIS  OF  THE  MODEL  FIT  USING  THE  PRE¬ 
DICTIONS  OF  THIS  MODEL;  ELSE  ZERO. 

If  Binomial  and  Poisson  v are  successful ,  than: 

ENTER  ONE  FOR  PREDICTIONS  BASED  ON  BINOMIAL,  OR  TWO  FOR  POISSON. 

End  if 


FIGURE  11-9.  MDLBAM  ANALYSES  OF  MODEL  FIT  PROMPTS 
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11.2.2.  Generalized  Poisson  Mode! 


If  the  entered  model  option  number  is  two,  then: 


a.  SMERFS  prompts  for  a  flag  indicating  whether  the  model  assumptions  and  data  requirements 
are  desired  (Figure  1 1-10).  If  desired,  the  listing  and  extra  prompts  in  the  figure  are  output. 
If  the  response  to  the  last  prompt  indicates  a  desire  to  abort  the  model  execution,  control 
returns  to  Section  1 1 .2,  step  a.  (Refer  to  Paragraph  1 1 .3.2  for  details  on  the  prompted  input 
and  generated  output.) 


PITER  ONE  FOR  GENERALIZED  POISSON  MODEL  DESCRIPTION;  ELSE  ZERO. 
THE  GENERALIZED  POISSON  MODEL  ASSUMPTIONS 

1.  THE  SOFTWARE  IS  OPERATED  IN  A  SIMILAR  MANNER  AS  THE  ANTICI¬ 
PATED  OPERATIONAL  USAGE. 

2.  ALL  FAULTS  ARE  EQUALLY  LIKELY  TO  OCCUR  AND  ARE  INDEPENDENT 
OF  EACH  OTHER. 

3.  EACH  FAULT  IS  OF  THE  SAME  ORDER  OF  “IMPACT"  AS  ANY  OTHER. 

4.  THE  EXPECTED  NUMBER  OF  FAULTS  OCCURRING  IN  ANY  TIME  INTER¬ 
VAL  IS  PROPORTIONAL  TO  THE  FAULT  CONTENT  AT  THE  TIME  OF 
TESTING,  AND  TO  SOME  FUNCTION  OF  THE  AMOUNT  OF  TIME  SPENT 
IN  TESTING. 

5.  FAULTS  ARE  CORRECTED  AT  THE  ENDS  OF  THE  TESTING  INTERVALS, 
WITHOUT  INTRODUCING  NEW  ONES. 

ENTER  ONE  TO  CONTINUE  THE  DESCRIPTION  LIST. 

MODEL  DATA  REQUIREMENTS 

1.  THE  NUMBER  OF  FAULTS  DETECTED  IN  EACH  INTERVAL  OF  TESTING. 

2.  THE  LENGTHS  OF  THE  VARIOUS  TESTING  INTERVALS. 

“I  &  2  SHOULD  HAVE  BEEN  ENTERED  VIA  THE  DATA  INPUT  MODULE." 

3.  THE  NUMBER  OF  FAULTS  CORRECTED  AT  THE  END  OF  EACH  TESTING 
PERIOD. 

IF  THE  ASSUMPTIONS  ARE  GENERALLY  SATISFIED  AND  THE  APPROPRIATE 
DATA  ARE  AVAILABLE,  ENTER  ONE  TO  CONTINUE  THE  MODEL;  OTHERWISE 
EERO  TO  RETURN  TO  THE  MENU  OF  MODELS. 


FIGURE  11-10.  MDLGPO  DESCRIPTION  PROMPTS  AND  LIST 


b.  SMERFS  prompts  for  a  flag  indicating  whether  fault  corrections  were  performed  in  the  same 
interval  in  which  they  were  detected  (Figure  11-11).  If  all  fault  detections  and  corrections 
occurred  in  the  same  intervals,  the  program  constructs  the  fault  correction  vector;  otherwise. 
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SMERFS  prompts  the  user  to  enter  the  number  corrected  at  die  end  of  each  testing  period, 
as  shown  in  the  second  prompt  of  the  figure. 


ENTER  OMB  IT  ANY  PAULXS  DETECTED  XM  A  GIVEN  TESTING  PERIOD  WERE 
CORRECTED  IN  A  LATER  PERIOD;  ELSE  SERO. 

If  a  positiva  rasponae  was  input,  than  a  marias  oft 

ENTER  NUMBER  CORRECTED  AT  THE  END  OP  PERIOD  iii. 

End  if 


FIGURE  11-11.  MDLGPO  CORRECTION  VECTOR  CREATION  PROMPTS 


c.  SMERFS  then  prompts  the  user  to  enter  the  desired  method  of  estimation  (ML  or  LS)  or  a 
value  to  terminate  this  model  execution  (Figure  1 1-12).  If  the  termination  value  is  entered, 
control  branches  to  seep  g.  If  a  method  of  estimation  is  selected,  then  SMERFS  prompts  for 
the  desired  Weighting  function  number.  If  an  invalid  option  number  is  entered,  die  same 
list  and  third  prompt  are  output.  The  output  of  the  third  option  only  occurs  during  a  ML 
estimation  execution  (in  which  the  testing  intervals  are  of  unequal  lengths).  Entry  of  a  three 
during  a  LS  estimation  or  a  ML  estimation  in  which  the  testing  intervals  are  all  equal  is 
treated  as  an  invalid  response. 

The  remaining  program  prompts  (and  required  user  inputs)  are  contingent  upon  the  method 
of  estimation  and  the  Weighting  function  number.  The  following  rules  apply: 

1.  For  the  Schick-Wolverton  method,  no  additional  inputs  are  required. 

2.  For  the  ALPHA  input  method,  the  value  for  ALPHA  is  prompted  (Figure  11-12). 

3.  For  the  ML  ALPHA  estimated  method,  the  initial  estimates  for  both  the  total  number 
of  faults  and  ALPHA  are  internally  established.  Those  values  are  then  output  and 
SMERFS  allows  the  user  to  override  the  values  to  ensure  that  the  global  maximum 
has  been  reached  (Figure  11-12).  Then  the  program  prompts  for  the  maximum 
number  of  convergence  iterations. 
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EMTKK  ONE  FOR  MAXIMUM  LIKELIHOOD  METHOD,  TWO  FOK  LEAST  SQUARES 
METHOD,  OX  THREE  TO  TERMINATE  MODEL  EXECUTION. 

ENTER  WEIGHTING  FUNCTION  NUMBER,  OR  XERO  FOR  A  LIST. 

THE  AVAILABLE  WEIGHTING  FUNCTIONS  ARE: 

1  X(I)  «*  2/2  ( SCHICK-WOLVERTON  MODEL) 

2  X(I)  **  ALPHA  (WHERE  ALPHA  IS  INPUT) 

It  tha  ML  method  was  aelected ,  then : 

If  tha  fating  lmngtha  arm  variable,  than: 

3  2(1)  **  ALPHA  (WHERE  ALPHA  IS  ESTIMATED) 

Elam,  if  tha  fating  lmngtha  arm  all  aqual,  than: 

*  ALPHA  CANNOT  BE  ESTIMATED  IF  THE  INTERVAL  LENGTHS  ARE  FIXED. 
End  if 
End  if 

ENTER  WEIGHTING  FUNCTION  NUMBER. 

It  tha  ALPHA  input  mathod  was  selected,  then: 

ENTER  DESIRED  ALPHA. 

Siam,  if  tha  ALPHA  aatimatad  mathod  was  aalactad,  than i 

THE  INITIAL  ESTIMATES  TO  BE  USED  IN  THE  ESTIMATION  PROCESS  ARE: 
TOTAL  NUMBER  OF  FAULTS  eeeeeeeeeee 
ALPHA  eeeeeeeeeee 

ENTER  ONE  TO  USE  DIFFERENT  INITIAL  ESTIMATES,  OR  SERO  TO  USE  THE 
INITIAL  ESTIMATES. 

If  uaer  override  was  selected,  then; 

ENTER  INITIAL  ESTIMATE  FOR  THE  TOTAL  NUMBER  OP  FAULTS 
(BETWEEN  eeeeeeeeeee  AMD  0.10000E+08) . 

ENTER  INITIAL  ESTIMATE  FOR  ALPHA 
(BETWEEN  0.00000E+00  AND  0 . 10000E+08)  . 

End  If 

ENTER  MAXIMUM  NUMBER  OF  ITERATIONS. 

End  if 


FIGURE  11-12.  MDLGPO  INPUT  PROMPTS 

d.  If  the  maximum  number  of  iterations  is  reached  before  a  solution  is  found,  the  Trust  Region 

could  not  be  adjusted  properly  using  the  initial  estimates  for  ALPHA  and  the  total  number 
of  faults,  the  data  are  not  appropriate  for  this  model,  or  the  estimate  of  the  total  number  of 
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faults  is  less  than  the  observed  number,  then  the  appropriate  message  shown  in  Figure  11-13 
(including  the  ML  or  LS  indicator  and  the  Weighting  function  number)  is  issued  and  control 
returns  to  step  c. 


••THE  MAXIMUM  NUMBER  OF  ITERATIONS  NAS  REACHED  WITH  THE  FOLLOWING 
FINAL  ML  MODEL  ESTIMATES,  USING  THE  WEIGHTING  FUNCTION  TYPE  3: 
PROPORTIONALITY  CONSTANT  tMeniMM 

TOTAL  NUMBER  OF  FAULTS  eeeeeeeeeee 

4  OF  FAULTS  REMAINING  eeeeeeeeeee 

AND  ALPHA  eeeeeeeeeee 

••THE  TRUST  REGION  COULD  NOT  BE  ADJUSTED  PROPERLY  FOR  THE  ML  MODEL 
USING  THE  WEIGHTING  FUNCTION  TYPE  3. 

••THE  DATA  ARE  NOT  APPROPRIATE  FOR  THE  M  MODEL,  USING  THE  WEIGH¬ 
TING  FUNCTION  TYPE  1. 

••THE  ESTIMATE  FOR  THE  TOTAL  NUMBER  OF  FAULTS  TO  BE  DETECTED  IN 
THE  PROGRAM  BEFORE  ALL  FAULTS  ARE  UNCOVERED  CAME  OUT  LESS  THAN 
THE  NUMBER  OF  FAULTS  FOUND  TO  DATE  WITHIN  THE  aa  MODEL,  USING 
THE  WEIGHTING  FUNCTION  TYPE  i. 


FIGURE  11-13.  MDLGPO  PROCESSING  ERROR  MESSAGES 


e.  Otherwise  (if  the  solution  is  obtained),  the  ML  or  LS  estimates  are  output,  including  die 
Weighting  function  number  (Figure  11-14).  Within  that  figure,  the  upper  report  (containing 
the  95-percent  confidence  intervals)  corresponds  to  the  ML  estimates  with  a  Weighting 
function  number  of  one  or  two.  The  middle  report  corresponds  to  the  LS  estimates  with  a 
Weighting  function  number  of  one  or  two.  The  lower  report  (containing  die  additional 
values  for  ALPHA  and  the  number  of  iterations  performed  to  obtain  those  estimates) 
corresponds  to  the  ML  estimates  with  a  Weighting  function  number  of  three. 


If  the  ML  method  ( other  than  ALPHA  estimated)  was  selected,  then: 

ML  MODEL  ESTIMATES,  USING  THE  WEIGHTING  FUNCTION  TYPE  i  ARE: 

(THE  APPROXIMATE  95%  CONFIDENCE  INTERVALS  APPEAR  IN  PARENTHESES) 
PROPORTIONALITY  CONSTANT  eeeeeeeeeee  (  eeeeeeeeee,  eeeeeeeeee) 
TOTAL  NUMBER  OF  FAULTS  aeeeeeeeeee  (  —a——,  eeeeeeeeee) 
#  OF  FAULTS  REMAINING  aeeeeeeaeee  (  eeeeeeeeee,  eeeeeeeeee) 

Else,  if  the  LS  method  was  selected,  then: 

LS  MODEL  ESTIMATES,  USING  THE  WEIGHTING  FUNCTION  TYPE  i  ARE: 
PROPORTIONALITY  CONSTANT  eeeeeeeeeee 
TOTAL  NUMBER  OF  FAULTS  eeeeeeeeeee 

4  OF  FAULTS  REMAINING  eeeeeeeeeee 


FIGURE  11-14.  MDLGPO  SUCCESSFUL  CONVERGENCE  OUTPUT 
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Siam,  it  tha  ML  method  (with  ALPHA  •atiaafd)  warn  a* lac tad ,  t hen: 

ML  MODEL  ESTIMATES,  USING 

THE  WEIGHTING  FUNCTION  TYPE  3,  AFTER 

ill  ITERATIONS  ARE: 

PROPORTIONALITY  CONSTANT 

eeeaeaaeaae 

TOTAL  NUMBER  OF  FAULTS 

aaaaaaaaaaa 

#  OF  FAULTS  REMAINING 

aaaaaaaaaaa 

AND  ALPHA 

aaaaaaaaaaa 

Bnd  it 

FIGURE  11-14.  MDLGPO  SUCCESSFUL  CONVERGENCE  OUTPUT  (Continued) 


f.  SMERFS  then  allows  for  the  processing  of  future  predictions  to  determine  the  expected 

performance  of  the  software  (Figure  1 1-15).  Upon  completion,  control  returns  to  step  c. 

NOTE  1:  The  SMERFS  program  attempts  to  minimize  the  number  of  prompted  values 
by  cycling  only  on  the  final  value  (i.e.,  the  value  entered  for  die  projected 
length  of  the  period  is  prompted  only  once;  the  value  for  the  number  of  faults 
to  discover  is  prompted  until  the  zero  is  entered).  If  changing  of  the  "fixed” 
value  is  desired,  a  zero  should  be  entered  to  end  die  current  predictions,  and 
then  the  same  prediction  option  re-selected.  Both  values  will  then  be  prompted. 

NOTE  2:  Certain  predictions  can  result  in  infinite  looping;  die  SMERFS  program  guards 
against  this  condition  through  an  internal  program  maximum.  If  that  maximum 
is  reached,  an  error  message  is  issued.  Additionally,  the  user  is  allowed  to 
change  that  maximum  if  it  is  felt  that  the  maximum  is  too  high  or  too  low 
(refer  to  the  second  predictions  of  the  figure). 

ERROR:  If  the  estimate  for  the  (rounded)  number  of  faults  remaining  is  less  than  one, 

these  future  predictions  are  not  allowed  and  the  message  shown  at  the  bottom 
of  the  figure  is  issued. 


THE  AVAILABLE  FUTURE  PREDICTIONS  ARE: 

1)  THE  NUMBER  OF  FAULTS  EXPECTED  IN  THE  NEXT  TESTING  PERIOD 

2)  THE  NUMBER  OF  PERIODS  NEEDED  TO  DISCOVER  THE  NEXT  M  FAULTS 
ENTER  PREDICTION  OPTION,  OR  ZERO  TO  END  PREDICTIONS. 

ENTER  PROJECTED  LENGTH  OF  THE  PERIOD,  OR  ZERO  TO  END. 

#  OF  FAULTS  EXPECTED  aeeeeeeeeee  (  eeeeeeeeee,  aeeeeeaeaa) 

ENTER  PROJECTED  LENGTH  OF  THE  PERIODS. 

ENTER  VALUE  OF  N  (BETWEEN  ONE  AND  aaaaaaaaaaa) ,  OR  ZERO  TO  END. 

#  OF  PERIODS  EXPECTED  aeeeaeeaaae 


FIGURE  11-15.  MDLGPO  FUTURE  PREDICTION  PROMPTS  AND  OUTPUT 
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If  ti>«  maximum  iteration*  (1000)  is  reached,  than: 

•*#  or  PERIODS  EXCEEDS  eeeeeeeeee*  (PROGRAM  MAXIMUM) 

ENTER  ONE  TO  CHANGE  THE  PROGRAM  MAXIMUM;  ELSE  ZERO. 

ENTER  THE  NEW  VALUE  FOR  THE  PROGRAM  MAXIMUM. 

End  if 

♦•FUTURE  PREDICTIONS  ARE  NOT  ALLOWED  BECAUSE  THE  ESTIMATE  FOR  THE 
(ROUNDED)  NUMBER  OF  FAULTS  REMAINING  IS  LESS  THAN  ONE. 


FIGURE  11-15.  MDLGPO  FUTURE  PREDICTION  PROMPTS  AND  OUTPUT  (Continued) 


g.  The  user  has  responded  to  the  prompt  in  step  c  with  a  value  equated  to  termination.  Before 
the  return  is  performed,  SMERFS  determines  if  the  last  execution  for  either  of  the  methods 
of  estimation  was  successful;  if  either  was,  it  allows  for  various  analyses  of  die  model  fit. 
This  is  shown  in  die  first  prompt  of  Figure  11-16.  Additionally,  if  both  the  ML  and  LS 
methods  were  successful,  the  second  prompt  is  issued,  allowing  the  user  to  select  the  set  of 
estimates  to  be  used  in  the  analyses.  Control  then  automatically  transfers  to  the  Analyses 
of  Model  Fit  sub-module  (Chapter  12).  Upon  completion,  control  returns  to  the  Interval 
Data  Model  Menu  (Section  11.2,  step  a). 


ENTER  ONE  TO  PERFORM  AN  ANALYSIS  OF  THE  MODEL  FIT  USING  THE  PRE¬ 
DICTIONS  OF  THIS  MODEL;  ELSE  ZERO. 

If  ML  and  LS  ware  successful/  then.* 

ENTER  ONE  FOR  PREDICTIONS  BASED  ON  MAXIMUM  L.iELIHOOD  METHOD,  OR 
TWO  FOR  LEAST  8QUARES  METHOD. 

End  it 


FIGURE  11-16.  MDLGPO  ANALYSES  OF  MODEL  FIT  PROMPTS 
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11.2.3. 


If  the  entered  model  option  number  is  three,  then: 

a.  SMERFS  prompts  for  a  flag  indicating  whether  the  model  assumptions  and  data  requirements 
are  desired  (Figure  1 1-17).  If  desired,  the  listing  and  extra  prompts  in  the  figure  are  output. 
If  the  response  to  the  last  prompt  indicates  a  desire  to  abort  the  model  execution,  control 
returns  to  Section  1 1.2,  step  a.  (Refer  to  Paragraph  1 1 .3.3  for  details  on  the  prompted  input 
and  generated  output.) 


PITER  ONE  FOR  MBPP  MODEL  DESCRIPTION;  ELSE  ZERO. 

THE  NON-HOMOGENEOUS  POISSON  MODEL  ASSUMPTIONS 

1.  THE  SOFTWARE  IS  OPERATED  IN  A  SIMILAR  MANNER  AS  THE  ANTICI¬ 
PATED  OPERATIONAL  USAGE. 

2.  ALL  FAULTS  ARE  EQUALLY  LIKELY  TO  OCCUR  AND  ARE  INDEPENDENT 
OF  EACH  OTHER. 

3.  EACH  FAULT  IS  OF  THE  SAME  ORDER  OF  "IMPACT"  AS  ANY  OTHER. 

4.  THE  CUMULATIVE  NUMBER  OF  FAULTS  DETECTED  AT  ANY  TIME  FOL¬ 
LOWS  A  POISSON  DISTRIBUTION  WITH  MEAN  M(T) .  THAT  MEAN  IS 
SUCH  THAT  THE  EXPECTED  NUMBER  OF  FAULTS  IN  ANY  SMALL  TIME 
INTERVAL  ABOUT  T  IS  PROPORTIONAL  TO  THE  NUMBER  OF  UNDETEC¬ 
TED  FAULTS  AT  TIME  T. 

5.  THE  MEAN  IS  ASSUMED  TO  BE  A  BOUNDED  NON-DECREASING  FUNCTION 
WITH  M(T)  APPROACHING  IN  THE  LIMIT,  "A"  (THE  EXPECTED  TOTAL 
NUMBER  OF  FAULTS  TO  BE,  EVENTUALLY,  DETECTED  IN  THE  TESTING 
PROCESS),  AS  THE  LENGTH  OF  TESTING  GOES  TO  INFINITY. 

ENTER  ONE  TO  CONTINUE  THE  DESCRIPTION  LIST. 

MODEL  DATA  REQUIREMENTS 

1.  THE  NUMBER  OF  FAULTS  DETECTED  IN  EACH  INTERVAL  OF  TESTING. 

2.  THE  LENGTHS  OF  THE  VARIOUS  TESTING  INTERVALS. 

"1  &  2  SHOULD  HAVE  BEEN  ENTERED  VIA  THE  DATA  INPUT  MODULE." 

IF  THE  ASSUMPTIONS  ARE  GENERALLY  SATISFIED  AMD  THE  APPROPRIATE 
DATA  ARE  AVAILABLE,  ENTER  ONE  TO  CONTINUE  THE  MODEL;  OTHERWISE 
ZERO  TO  RETURN  TO  THE  MENU  OF  MODELS. 


FIGURE  11-17.  MDLNPI  DESCRIPTION  PROMPTS  AND  LIST 


b.  SMERFS  then  prompts  the  user  to  enter  the  desired  method  of  estimation  (ML  or  LS)  or  a 
value  to  terminate  the  model  execution  (Figure  1 1-18).  If  the  termination  value  is  entered, 
control  branches  to  step  f. 


11-15 


NSWCDD  TR  84-373 


PITER  ONE  FOR  MAXIMUM  LIKELIHOOD  METHOD,  TWO  FOR  LEAST  SQUARES 
METHOD,  OR  THREE  TO  TERMINATE  MODEL  EXECUTION. 


FIGURE  11-18.  MDLNPI  INPUT  PROMPT 


c.  If  die  data  are  not  appropriate  for  this  model  or  the  estimate  of  the  total  number  of  faults  is 
less  than  the  observed  number,  then  the  appropriate  message  shown  in  Figure  11*19 
(including  the  ML  or  LS  indicator)  is  issued  and  control  returns  to  step  b. 


**THE  DATA  ARE  NOT  APPROPRIATE  FOR  THE  aa  MODEL. 

••THE  ESTIMATE  FOR  THE  TOTAL  NUMBER  OF  FAULTS  TO  HE  DETECTED  IN 
THE  PROGRAM  BEFORE  ALL  FAULTS  ARE  UNCOVERED  CAME  OUT  LESS  THAN 
THE  NUMBER  OF  FAULTS  FOUND  TO  DATE  WITHIN  THE  aa  MODEL. 


FIGURE  11-19.  MDLNPI  PROCESSING  ERROR  MESSAGES 


d.  Otherwise  (if  the  solution  is  obtained),  the  ML  or  LS  estimates  are  output  (Figure  1 1*20). 
Within  that  figure,  the  upper  report  (containing  the  95-percent  confidence  intervals) 
corresponds  to  the  ML  estimates  and  the  lower  report  to  the  LS  estimates. 


It the  ML  method  vas  selected,  thent 
ML  MODEL  ESTIMATES  ARE: 

(THE  APPROXIMATE  95%  CONFIDENCE  INTERVALS  APPEAR  IN  PARENTHESES) 
PROPORTIONALITY  CONSTANT  MtMMaeM  (  Mamma,  aaaaaaam) 
TOTAL  NUMBER  OF  FAULTS  aaMMaam  (  MBMeem,  aoaaaaeeea) 

#  OF  FAULTS  REMAINING  eaeeeaeeeee  (  eooeaaaaaa,  aaaaaaaaaa) 

Elme ,  it  t he  LS  method  was  selected ,  then: 

LS  MODEL  ESTIMATES  ARE: 

PROPORTIONALITY  CONSTANT  eeeeaeeaeee 
TOTAL  NUMBER  OF  FAULTS  aeaaaeaeeea 

#  OF  FAULTS  REMAINING  eeaaaeaeeae 

End  it 


FIGURE  11-20.  MDLNPI  SUCCESSFUL  CONVERGENCE  OUTPUT 


e.  SMERFS  then  allows  for  the  processing  of  future  predictions  to  determine  the  expected 
performance  of  the  software  (Figure  1 1-21).  Upon  completion,  control  returns  to  step  b. 
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NOTE  1 :  The  SMERFS  program  attempts  to  minimize  the  number  of  prompted  values 
by  cycling  only  on  the  final  value  (i.e.,  the  value  entered  for  the  projected 
length  of  the  period  is  prompted  only  once;  the  value  for  the  number  of  faults 
to  discover  is  prompted  until  the  zero  is  entered).  If  changing  of  the  "fixed’ 
value  is  desired,  a  zero  should  be  entered  to  end  the  current  predictions,  and 
then  the  same  prediction  option  re-selected.  Both  values  will  then  be  prompted. 

NOTE  2:  Certain  predictions  can  result  in  infinite  looping;  the  SMERFS  program  guards 
against  this  condition  through  an  internal  program  maximum.  If  that  maximum 
is  reached,  an  error  message  is  issued.  Additionally,  the  user  is  allowed  to 
change  that  maximum  if  it  is  felt  that  the  maximum  is  too  high  or  too  low 
(refer  to  the  second  predictions  of  the  figure). 

ERROR:  If  the  estimate  for  the  (rounded)  number  of  faults  remaining  is  less  than  one, 

these  future  predictions  are  not  allowed  and  the  message  shown  at  the  bottom 
of  the  figure  is  issued. 


THE  AVAILABLE  FUTURE  PREDICTIONS  ARE: 

1)  THE  NUMBER  OF  FAULTS  EXPECTED  IN  THE  NEXT  TESTING  PERIOD 

2)  THE  NUMBER  OF  PERIODS  NEEDED  TO  DISCOVER  THE  NEXT  M  FAULTS 
ENTER  PREDICTION  OPTION,  OR  ZERO  TO  END  PREDICTIONS. 

ENTER  PROJECTED  LENGTH  OF  THE  PERIOD,  OR  ZERO  TO  END. 

#  OF  FAULTS  EXPECTED  eeeeeeeeeee  (  eeeeeeeeee,  eeeeeeeeee) 

ENTER  PROJECTED  LENGTH  OF  THE  PERIODS. 

ENTER  VALUE  OF  N  (BETWEEN  ONE  AND  eeeeeeeeeee),  OR  ZERO  TO  END. 

#  OF  PERIODS  EXPECTED  eeeeeeeeeee 

If  the  maximum  iterations  (1000)  is  reached,  them 

**#  OF  PERIODS  EXCEEDS  eeeeeeeeeee  (PROGRAM  MAXIMUM) 

ENTER  ONE  TO  CHANGE  THE  PROGRAM  MAXIMUM;  ELSE  ZERO. 

ENTER  THE  NEW  VALUE  FOR  THE  PROGRAM  MAXIMUM. 

End  if 

** FUTURE  PREDICTIONS  ARE  NOT  ALLOWED  BECAUSE  THE  ESTIMATE  FOR  THE 
(ROUNDED)  NUMBER  OF  FAULTS  REMAINING  IS  LESS  THAN  ONE. 


FIGURE  11-21.  MDLNPI  FUTURE  PREDICTION  PROMPTS  AND  OUTPUT 


f.  The  user  has  responded  to  the  prompt  in  step  b  with  a  value  equated  to  termination.  Before 
the  return  is  performed,  SMERFS  determines  if  the  last  execution  for  either  of  the  methods 
of  estimation  was  successful;  if  either  was,  it  allows  for  various  analyses  of  the  model  fit. 
This  is  shown  in  the  first  prompt  of  Figure  11-22.  Additionally,  if  both  the  ML  and  LS 
methods  were  successful,  the  second  prompt  is  issued,  allowing  the  user  to  select  the  set  of 
estimates  to  be  used  in  the  analyses.  Control  then  automatically  transfers  to  the  Analyses 
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of  Model  Fit  sub-module  (Chapter  12).  Upon  completion,  control  returns  to  the  Interval 
Data  Model  Menu  (Section  11.2,  step  a). 


mm  ONE  TO  PERFORM  AM  ANALYSIS  OP  THE  MODEL  PIT  USING  THE  PRE¬ 
DICTIONS  OP  THIS  MODEL;  ELSE  ZERO. 

It  ML  and  LS  vara  successful ,  than: 

ENTER  ONE  FOR  PREDICTIONS  BASED  ON  MAXIMUM  LIKELIHOOD  METHOD,  CUR 
TWO  FOR  LEAST  SQUARES  METHOD. 

Bnd  it 


FIGURE  11-22.  MDLNPI  ANALYSES  OF  MODEL  FIT  PROMPTS 
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11.2.4  Schneidewind*s  Maximum  Likelihood  Model 
If  the  entered  model  option  number  is  four,  then: 

a.  SMERFS  prompts  for  a  flag  indicating  whether  the  model  assumptions  and  date  requirements 
are  desired  (Figure  1 1-23).  If  desired,  the  listing  and  extra  prompts  in  the  figure  are  output. 
If  the  response  to  the  last  prompt  indicates  a  desire  to  abort  the  model  execution,  control 
returns  to  Section  1 1 .2,  step  a.  (Refer  to  Paragraph  1 1 .3.4  for  details  on  the  prompted  input 
and  generated  output.) 


FIGURE  11-23.  MDLSDW  DESCRIPTION  PROMPTS  AND  LIST 


b.  SMERFS  then  permits  a  description  of  the  three  Treatment  types  for  this  model  by  issuing 
the  prompt  shown  in  Figure  11-24.  If  the  response  entered  indicates  that  the  additional 
information  is  desired,  the  list  is  output. 


11-19 


NSWCDD  TR  84-373 


FIGURE  11-24.  MDLSDW  TREATMENT  DESCRIPTION  PROMPT  AND  LIST 


c.  SMERFS  then  allows  for  the  investigation  of  the  "optimum  S"  (Reference  6)  through  the 
first  prompt  of  Figure  1 1-25.  If  the  response  entered  indicates  that  this  additional  analysis 
is  desired,  then  SMERFS  prompts  for  the  range  over  which  S  is  to  be  processed.  As  shown 
in  the  figure,  a  table  of  the  resulting  ALPHA,  BETA,  WLS,  MSEp,  and  MSEp  values  for 
each  value  of  S  within  the  range  will  be  generated.  (Note,  if  the  calculation  for  the  MSEp 
results  in  the  attempt  to  take  the  log  of  a  non-positive  number,  that  calculation  is  terminated 
and  the  field  is  left  blank.)  The  optimum  S  based  on  the  weighted  sums-of-squares  is  the 
one  resulting  in  the  smallest  value.  The  optimum  S  for  either  MSEp  or  MSEp  (the  preferred 
methods)  is  the  one  resulting  also  in  the  smallest  value  for  the  chosen  criteria.  (Refer  to 
Paragraph  11.3.4  for  specific  information  on  this  selection  process.) 


ENTER  ONE  TO  INVESTIGATE  FOR  THE  OPTIMUM  S  (USING  TREATMENT  TYPE 
NUMBER  2);  ELSE  ZERO  TO  CONTINUE  WITH  THE  MODEL  EXECUTION. 

If  an  affirmative  response  was  entered,  then: 

ENTER  RANGE  OVER  WHICH  S  SHOULD  BE  TESTED.  NOTE,  AN  EXECUTION 
ON  A  GIVEN  S  WHICH  FAILED  THE  CONVERGENCE  CRITERIA  WILL  NOT  BE 
INCLUDED  IN  THE  FOLLOWING  RESULTS  TABLE.  THE  OPTIMUM  S  FOR  EI¬ 
THER  MSE-F  OR  MSE-T  IS  THE  ONE  RESULTING  IN  THE  SMALLEST  VALUE 
FOR  YOUR  CHOSEN  CRITERIA. 

S  BETA  ALPHA  WLS  MSE-F  MSE-T 


ill  eeeeeeeeeee  eeeeeeeeeee  eeeeeeeeeee  eeeeeeeeeee  eeeeeeeeeee 


ill  eeeeeeeeeee  eeeeeeeeeee  eeeeeeeeeee  eeeeeeeeeee  eeee 
ENTER  ONE  TO  INVESTIGATE  ANOTHER  RANGE  FOR  S;  ELSE  ZERO. 
End  if 


FIGURE  11-25.  MDLSDW  OPTIMUM  S  PROMPTS  AND  OUTPUT 
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d.  SMERFS  prompts  the  user  to  enter  the  desired  Treatment  number  or  a  value  to  terminate 
this  model  execution  (Figure  11-26).  If  the  termination  value  is  entered,  control  branches 
to  step  h.  If  a  Treatment  type  is  selected,  SMERFS  prompts  for  the  value  of  the  variable 
"S"  (if  the  Treatment  type  was  two  or  three). 


ENTER  DESIRED  MODEL  TREATMENT  NUMBER,  OR  FOUR  TO  TERMINATE  MODEL 
EXECUTION. 

It  the  Treatment  type  is  two  or  three,  then: 

ENTER  ASSOCIATED  VALUE  OF  S  (LESS  THAN  THE  NUMBER  OF  PERIODS). 
End  it 


FIGURE  11-26.  MDLSDW  INPUT  PROMPTS 


e.  If  the  data  are  not  appropriate  for  this  model  or  the  estimate  of  the  total  number  of  faults  is 
less  than  the  observed  number,  then  the  message  shown  in  Figure  11-27  (including  the 
Treatment  type  number)  is  issued  and  control  returns  to  step  d. 


♦•THE  DATA  ARE  NOT  APPROPRIATE  FOR  THE  TREATMENT  NUMBER  i  MODEL. 

♦♦THE  ESTIMATE  FOR  THE  TOTAL  NUMBER  OF  FAULTS  TO  BE  DETECTED  IN 
THE  PROGRAM  BEFORE  ALL  FAULTS  ARE  UNCOVERED  CAME  OUT  LESS  THAN 
THE  NUMBER  OF  FAULTS  FOUND  TO  DATE  WITHIN  THE  TREATMENT  NUMBER  i 
MODEL. 


FIGURE  11-27.  MDLSDW  PROCESSING  ERROR  MESSAGES 


f.  Otherwise  (if  the  solution  is  obtained),  the  estimates  are  output,  including  the  Treatment  type 
number  (Figure  11-28). 


TREATMENT  1  MODEL  ESTIMATES  ARE: 

BETA  eeeeeeeeeee 

ALPHA  eeeeeeeeeee 

TOTAL  NUMBER  OF  FAULTS  eeeeeeeeeee 

If  the  Treatment  type  is  two,  then: 

PLUS  THOSE  SKIPPED  eeeeeeeeeee  IN  PERIODS  1  THROUGH  ill 

End  it 


FIGURE  11-28.  MDLSDW  SUCCESSFUL  CONVERGENCE  OUTPUT 
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4  OF  FAULTS  REMAINING 
WEIGHTED  SUMS-OF-SQUARES 
BETWEEN  PREDICTED  AND 
OBSERVED  FAULTS 
MEAN  SQUARE  ERROR  FOR 
CUMULATIVE  FAULTS 


It  til*  calculation  was  not  ended  by  a  non-poaitive  value,  then: 

MEAN  SQUARE  ERROR  FOR 
TIME  TO  NEXT  FAILURE 

End  it 


FIGURE  11-28.  MDLSDW  SUCCESSFUL  CONVERGENCE  OUTPUT  (Continued) 


g.  SMERFS  then  allows  for  the  processing  of  future  predictions  to  determine  the  expected 
performance  of  the  software  (Figure  11-29).  Upon  completion,  control  returns  to  step  d. 

NOTE:  If  the  prediction  for  the  number  of  periods  cannot  be  made  for  the  entered 

value  for  the  number  of  faults,  the  shown  error  message  is  issued. 

ERROR:  If  the  estimate  for  the  (rounded)  number  of  faults  remaining  is  less  than  one, 

these  future  predictions  are  not  allowed  and  the  message  shown  at  the  bottom 
of  the  figure  is  issued. 


THE  AVAILABLE  FUTURE  PREDICTIONS  ARE: 

1)  THE  NUMBER  OF  FAULTS  EXPECTED  IN  THE  NEXT  TESTING  PERIOD 

2)  THE  NUMBER  OF  PERIODS  NEEDED  TO  DISCOVER  THE  NEXT  M  FAULTS 
ENTER  PREDICTION  OPTION,  OR  ZERO  TO  END  PREDICTIONS. 

ENTER  NUMBER  OF  PERIODS  TO  EXAMINE,  OR  ZERO  TO  END. 

#  OF  FAULTS  EXPECTED  •aeeeeeaeee 

ENTER  VALUE  OF  M  (BETWEEN  ONE  AND  aaaaaaaaaaa ) ,  OR  ZERO  TO  END. 

#  OF  PERIODS  EXPECTED  eeeeeeeaeae 

It  tha  prediction  could  not  be  made  for  the  input  M  value,  then: 

**  THE  ESTIMATE  CANNOT  BE  MADE  FOR  THE  SPECIFIED  M  VALUE. 


End  if 

♦•FUTURE  PREDICTIONS  ARE  NOT  ALLOWED  BECAUSE  THE  ESTIMATE  FOR  THE 
(ROUNDED)  NUMBER  OF  FAULTS  REMAINING  IS  LESS  THAN  ONE. 


FIGURE  11-29.  MDLSDW  FUTURE  PREDICTION  PROMPTS  AND  OUTPUT 
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b.  The  user  has  responded  to  the  prompt  in  step  d  with  a  value  equated  to  termination.  Before 
the  return  is  performed,  SMERFS  determines  if  the  last  execution  of  any  of  the  three 
Treatment  types  was  successful;  if  any  were,  it  allows  for  various  analyses  of  the  model  fit. 
This  is  shown  in  the  first  prompt  of  Figure  1 1-30.  Additionally,  if  more  than  one  Treatment 
type  were  successful,  the  second  prompt  is  issued,  allowing  the  user  to  select  the  set  of 
estimates  to  be  used  in  the  analyses.  Control  then  automatically  transfers  to  the  Analyses 
of  Model  Fit  sub-module  (Chapter  12).  Upon  completion,  control  returns  to  the  Interval 
Data  Model  Menu  (Section  11.2,  step  a). 

NOTE:  For  Treatment  type  two,  the  first  S-l  intervals  are  eliminated  from  the  observed 

data  vector.  For  treatment  type  three,  the  vector  is  reduced  by  S-2  predictions, 
where  the  first  S-l  intervals  (of  observed  fault  counts)  are  summed  on  interval 
one,  and  intervals  two  through  S-l  are  eliminated  from  the  observed  data 
vector.  This  is  reflected  by  the  final  message  of  the  figure. 


FIGURE  11-30.  MDLSDW  ANALYSES  OF  MODEL  FIT  PROMPTS 
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11.2.5  S-Shaoed  Reliability  Growth  Model 

If  die  entered  model  option  number  is  five,  then: 

a.  SMERFS  prompts  for  a  flag  indicating  whether  the  model  assumptions  and  data  requirements 

are  desired  (Figure  1 1-31).  If  desired,  the  listing  and  extra  prompts  in  the  figure  are  output. 
If  the  response  to  the  last  prompt  indicates  a  desire  to  abort  the  model  execution,  control 
returns  to  Section  11.2,  step  a.  (Refer  to  Paragraph  11.3.5  for  details  on  the  generated 
output.) 


iwu  om  for  s-shaped  model  description;  else  zero. 

THE  S-SHAPED  RELIABILITY  GROWTH  MODEL  ASSUMPTIONS 

1.  THE  SOFTWARE  IS  OPERATED  IN  A  SIMILAR  MANNER  AS  THE  ANTICI¬ 
PATED  OPERATIONAL  USAGE. 

2.  ALL  FAULTS  ARE  EQUALLY  LIKELY  TO  OCCUR  AND  ARE  INDEPENDENT 
OF  EACH  OTHER. 

3.  EACH  FAULT  IS  OF  THE  SAME  ORDER  OF  "IMPACT"  AS  ANY  OTHER. 

4.  THE  INITIAL  FAULT  CONTENT  OF  THE  SOFTWARE  SYSTEM  IS  A 
llll  RANDOM  VARIABLE . 

5.  THE  TIME  BETWEEN  FAILURES  (K  -  1)  AND  K  DEPENDS  ON  THE  TIME 
TO  FAILURE  (K  -  1). 

6.  FAULTS  ARE  CORRECTED  IMMEDIATELY  WITHOUT  INTRODUCING  NEW 
ONES. 

ENTER  ONE  TO  CONTINUE  THE  DESCRIPTION  LIST. 

MODEL  DATA  REQUIREMENTS 

1.  THE  NUMBER  OF  FAULTS  DETECTED  IN  EACH  INTERVAL  OF  TESTING. 

2.  THE  LENGTHS  OF  THE  VARIOUS  TESTING  INTERVALS. 

"1X2  SHOULD  HAVE  BEEN  ENTERED  VIA  THE  DATA  INPUT  MODULE." 

IF  TEE  ASSUMPTIONS  ARE  GENERALLY  SATISFIED  AND  TEE  APPROPRIATE 
SUITA  ARE  AVAILABLE,  ENTER  ONE  TO  CONTINUE  THE  MODEL;  OTHERWISE 
EERO  TO  RETURN  TO  THE  MENU  OF  MODELS. 


FIGURE  11-31.  MDLESH  DESCRIPTION  PROMPTS  AND  UST 


b.  If  die  data  are  not  appropriate  for  this  model  or  the  estimate  of  die  total  number  of  faults  is 
less  than  the  observed  number,  then  the  message  shown  in  Figure  1 1-32  is  issued  and  control 
brandies  to  step  e. 
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••THE  DATA  ARE  NOT  APPROPRIATE  FOR  THE  MODEL. 

••THE  ESTIMATE  FOR  THE  TOTAL  NUMBER  OF  FAULTS  TO  BE  DETECTED  IN 
THE  PROGRAM  BEFORE  ALL  FAULTS  ARE  UNCOVERED  CAME  OUT  LESS  THAN 
THE  NUMBER  OF  FAULTS  FOUND  TO  DATE  WITHIN  THE  MODEL. 


FIGURE  11-32.  MDLESH  PROCESSING  ERROR  MESSAGES 


c.  Otherwise  (if  the  solution  is  obtained),  the  estimates  are  output  (Figure  11-33). 


FIGURE  11-33.  MDLESH  SUCCESSFUL  CONVERGENCE  OUTPUT 


d.  SMERFS  then  allows  for  the  processing  of  future  predictions  to  determine  the  expected 
performance  of  the  software  (Figure  11-34). 

NOTE:  Certain  predictions  can  result  in  infinite  looping;  the  SMERFS  program  guards 

against  this  condition  through  an  internal  program  maximum.  If  that  maximum 
is  reached,  an  error  message  is  issued.  Additionally,  the  user  is  allowed  to 
change  that  maximum  if  it  is  felt  that  the  maximum  is  too  high  or  too  low 
(refer  to  the  second  predictions  of  the  figure). 

ERROR:  If  the  estimate  for  the  (rounded)  number  of  faults  remaining  is  less  than  one, 

these  future  predictions  are  not  allowed  and  the  message  shown  at  the  bottom 
of  the  figure  is  issued. 


THE  AVAILABLE  FUTURE  PREDICTIONS  ARE: 

1)  THE  NUMBER  OF  FAULTS  EXPECTED  IN  THE  NEXT  TESTING  PERIOD 

2)  THE  NUMBER  OF  PERIODS  NEEDED  TO  DISCOVER  THE  NEXT  M  FAULTS 

3)  THE  SOFTWARE  RELIABILITY  IN  THE  NEXT  PERIOD 
ENTER  PREDICTION  OPTION,  OR  ZERO  TO  END  PREDICTIONS. 

ENTER  PROJECTED  LENGTH  OF  THE  PERIOD,  OR  ZERO  TO  END. 

#  OF  FAULTS  EXPECTED  eeeeeeeeeee 


FIGURE  11-34.  MDLESH  FUTURE  PREDICTION  PROMPTS  AND  OUTPUT 
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sum  PROJECTED  LENGTH  OP  THE  PERIODS. 

ENTER  VALUE  OP  N  (BETWEEN  ONE  AND  •••••••••••) ,  OR  EERO  TO  BED. 

#  OP  PERIODS  EXPECTED  —— —9— 

Zf  t A*  maximum  iterations  (1000)  is  reached,  than; 

•*#  OP  PERIODS  EXCEEDS  aaaoaaaa—  (PROGRAM  MAXIMUM) 

ENTER  ONE  TO  CHANGE  THE  PROGRAM  MAXIMUM;  ELSE  ZERO. 

ENTER  THE  NEW  VALUE  FOR  THE  PROGRAM  MAXIMUM. 

End  if 

ENTER  PROJECTED  LENGTH  OP  THE  PERIOD ,  OR  ZERO  TO  END. 

PROB.  OF  OPERATION  WITH¬ 
OUT  FAILURE  IN  PERIOD  MMttWH  * 

** FUTURE  PREDICTIONS  ARE  NOT  ALLOWED  BECAUSE  THE  ESTIMATE  FOR  THE 
(ROUNDED)  NUMBER  OP  FAULTS  REMAINING  IS  LESS  THAN  ONE. 


FIGURE  11-34.  MDLESH  FUTURE  PREDICTION  PROMPTS  AND  OUTPUT  (Continued) 


e.  SMERFS  determines  if  the  execution  was  successful.  If  it  was,  it  allows  for  various 
analyses  of  the  model  fit.  This  is  shown  in  the  prompt  of  Figure  11-35.  Control  then 
automatically  transfers  to  the  Analyses  of  Model  Fit  sub-module  (Chapter  12).  Upon 
completion,  control  returns  to  the  Interval  Data  Model  Menu  (Section  11.2,  step  a). 


ENTER  ONE  TO  PERFORM  AN  ANALYSIS  OP  THE  MODEL  PIT  USING  THE  PRE¬ 
DICTIONS  OF  THIS  MODEL;  ELSE  ZERO. 


FIGURE  11-35.  MDLESH  ANALYSES  OF  MODEL  FIT  PROMPT 
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11.3  MODEL  OUTPUTS  AND  CONSIDERATIONS 

Hie  reader  is  directed  to  the  references  for  a  detailed  description  of  the  model  parameters  and 
outputs  covered  in  this  chapter.  Reference  2  is  especially  applicable  unless  indicated  otherwise  in  this 
section.  This  section  (as  did  Section  10.3),  discusses  the  meaning  of  any  required  inputs  and  displayed 
outputs.  Considerations  for  requested  inputs  are  also  discussed.  One  of  the  biggest  changes  from 
SMERFS  version  3  to  4  (thus  incorporated  in  version  5)  is  that  most  of  the  models  no  longer  require  the 
input  of  die  starting  values  for  the  optimization  procedures  for  model  parameter  estimation.  The  models 
using  die  Dekker-Brent  optimization  procedure  have  built-in  calculations  for  such  values  and,  as  a  result, 
no  user  input  is  required.  The  only  models  not  employing  this  technique  in  this  section  are  die 
Generalized  Poisson  Model  and  the  Brooks  and  Motley’s  Discrete  Software  Reliability  Model.  These 
models  employ  the  Trust  Region  methodology.  This  technique  will  still,  however,  determine  starting 
values  or  let  the  user  override  them  and  input  different  starting  values. 


U.3.1  Brooks  and  Motley’s  Discrete  Software  Reliability  Model 

If  the  user  indicates  to  the  program  that  not  alt  of  the  code  was  under  test  in  each  test  period,  the 
user  will  have  to  input  two  additional  quantities  for  each  test  period.  The  first  is  the  fraction  of  the  code 
being  tested  for  the  given  period,  and  the  second  is  the  cumulative  total  number  of  faults  detected  in  this 
fractional  part  over  the  past  testing  periods.  (Note  that  for  the  first  testing  period,  only  the  fraction  of 
code  under  test  is  requested  as  the  value  for  the  number  of  faults  detected  previously  in  this  section  of 
the  code  is  set  automatically  by  the  program  to  zero.) 

A  suggested  range  of  values  for  ALPHA  (the  probability  of  correcting  faults  in  the  program 
without  inserting  new  ones)  is  0.85  to  0.95  if  no  prior  knowledge  is  available.  If  the  user  decides  to 
override  the  suggested  initial  estimates  for  the  probability  of  detecting  faults  and  the  total  number  of 
faults,  the  user  estimates  must  be  in  the  ranges  1.0E-07  to  9.99E-01  and  the  total  number  of  faults  found 
to  date  to  1.0E+07,  respectively. 

The  program  gives  estimates  of  the  total  number  of  faults,  the  number  of  remaining  faults,  and  the 
probability  of  fault  detection.  In  addition,  the  program  allows  the  user  to  estimate  the  expected  number 
of  faults  in  the  next  testing  period  and  the  number  of  periods  required  to  discover  the  next  M  faults 
(Figure  11-8).  For  the  first,  the  user  must  enter  the  length  of  the  testing  period  (see  Paragraph  11.3.3) 
and  the  fraction  of  the  code  that  will  be  tested.  For  example,  if  a  T  was  entered  in  the  original  data 
entry  to  represent  a  month’s  worth  of  testing  and  the  user  is  particularly  interested  in  a  2-week  testing 
period,  then  enter  2/4  =  0.5.  If  the  full  program  was  to  be  under  test,  enter  a  ’’l”  for  the  fractional  part 
of  the  software.  (If  this  value  is  entered,  the  user  should  enter  the  total  number  of  faults  that  have  been 
found  to  date  for  the  entire  program,  after  the  prompt  asking  for  the  number  of  faults  found  in  this 
section  of  code  under  test  is  given.) 

For  the  second  prediction,  the  user  is  again  prompted  for  the  length  of  the  testing  periods,  the 
fraction  of  code  that  will  be  tested,  and  the  number  of  faults  that  have  been  found  to  date  in  this  section 
of  code,  as  well  as  the  desired  number  of  faults  to  be  found.  The  same  comments  as  given  for  the  first 
prediction  estimate  apply  for  this  one  as  well. 
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11.3.2  Generalized  Poisson  Model 

The  first  prompt,  after  the  model  assumptions,  asks  if  any  faults  detected  in  one  testing  period  were 
corrected  in  a  later  period.  For  example,  five  faults  were  detected  in  the  first  period,  but  only  three  of 
them  were  corrected  during  this  time  frame.  If  this  is  the  situation,  the  user  enters  the  number  of  faults 
actually  corrected  during  each  testing  period,  following  the  prompt  shown  in  Figure  11-11  by  die 
computer. 

The  user  then  has  air  option  for  the  type  of  weighting  function  to  assign  to  the  lengths,  [X(I>]  of 
die  respective  testing  periods  (Figure  11-12  and  Reference  2).  If  the  ALPHA  input  option  is  selected, 
die  larger  the  value  of  ALPHA,  the  greater  the  weight  applied  to  the  testing  period.  Negative  values  for 
ALPHA  also  work.  Usually,  a  value  of  1.0  will  be  tried  first,  followed  by  2.0,  0.S,  0.1,  and  finally 
-1.0.  If  die  user  desires  to  estimate  ALPHA  (increasing  the  number  of  estimated  parameters  to  3),  the 
data  base  should  be  fairly  large  (SO  or  more  values).  If  ALPHA  is  to  be  estimated,  the  Trust  procedure 
will  be  used,  necessitating  the  user  to  make  a  decision  whether  to  use  or  override  initial  program 
calculated  estimates  for  the  total  number  of  faults  and  ALPHA.  The  estimate  for  die  total  number  of 
faults  should  be  larger  than  die  number  found  to  date  and  less  than  1.0E+07.  The  initial  estimate  for 
ALPHA  must  be  between  0.0  and  1.0E+07. 

The  program  gives  estimates  of  the  total  number  of  faults,  the  number  of  remaining  faults,  ALPHA 
if  estimated,  and  the  proportionality  constant  of  the  model,  along  with  the  associated  95-percent 
confidence  intervals.  The  user  has  the  option  of  predicting  the  number  of  faults  expected  in  the  next 
testing  period  and/or  the  number  of  periods  needed  to  discover  the  next  M  faults.  In  either  case  the  user 
must  enter  the  projected  length  of  the  testing  period.  As  before,  if  "l"  was  entered  in  die  original  data 
set  to  represent  a  month’s  worth  of  testing  and  the  user  is  interested  in  a  3-month  period,  the  user  enters 
"3".  The  one  prediction  will  give  how  many  faults  are  estimated  to  be  found  during  this  time  period, 
while  the  other  will  give  how  many  periods  of  the  entered  length  are  required  to  find  the  next  M  (user 
input)  faults. 


11.3.3  Non-homoeeneous  Poisson  Model  for  Interval  Data 

The  program  gives  estimates  of  the  proportionality  constant,  the  total  number  of  faults,  and  die 
number  of  remaining  faults,  along  with  the  associated  95-percent  confidence  intervals.  For  prediction 
purposes,  the  user  can  estimate  the  number  of  faults  expected  in  the  next  period  and/or  the  number  of 
periods  required  to  discover  the  next  M  faults.  The  same  comments  as  applied  to  the  Brooks  and  Motley 
models  (Paragraph  1 1 .3. 1)  and  the  Generalized  Poisson  model  (Paragraph  1 1 .3.2)  on  the  meaning  of  the 
required  inputs  for  these  two  prediction  estimates  apply  here. 


11.3.4  Schneidewind’s  Maximum  Likelihood  Model 

The  user  must  select  the  appropriate  Treatment  type  and  if  type  2  or  3  is  selected,  a  value  for  "S" 
must  be  entered.  The  value  for  S  determines  how  much  weight  should  be  placed  upon  the  initial  testing 
periods.  If  the  user  feels  that  the  earlier  testing  periods  are  no  different  than  the  later  ones.  Treatment 
number  1  should  be  picked.  If,  however,  the  periods  are  significantly  different  (e.g.,  too  few  faults 
detected,  learning  curve  phenomena  observed,  etc.),  then  the  data  can  be  treated  in  one  of  two  ways. 
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The  earlier  testing  periods  can  be  either  discarded  by  using  Treatment  number  2  or  combined  by  using 
Treatment  number  3.  The  value  of  S  determines  this  transition  point. 

To  help  determine  the  optimal  value  of  S,  two  selection  criteria  have  been  incorporated  into  this 
new  version  of  SMERFS  to  reflect  Dr.  Schneidewind’s  recent  work  in  the  area  (Reference  9).  The 
criteria  are  the  Mean-Square-Error  for  cumulative  faults  (MSEp)  and  for  time  to  next  failure  (MSEr). 
Both  look  at  the  average  squared  error  difference  between  functions  of  the  actual  data  and  those  values 
estimated  by  the  model.  One  uses  the  actual  cumulative  fault  counts  and  those  predicted  by  die  model 
(MSEp).  The  other  uses  the  predicted  future  failure  counts  and  those  generated  by  the  model  (MSEr). 
Ideally  for  either  of  the  selected  criteria,  one  is  looking  for  the  optimal  value  of  S  that  minimizes  these 
differences.  The  user  needs  to  ensure,  by  varying  the  range  of  S  values,  that  the  optimal  S  has  been 
achieved;  however.  Dr.  Schneidewind’s  work  has  shown  that  the  first  local  minimum  found  may  be 
satisfactory.  The  weighted  sums-of-squares  is  a  third  criteria  that  can  also  be  used  to  determine  the 
optimal  S.  The  optimal  value  for  this  criteria  is  the  one  that  minimizes  the  weighted  least  squares 
function.  Of  the  three  criteria,  the  preferred  method  is  to  use  either  the  MSEp  or  the  MSEp. 

For  the  selected  model  and  associated  value  of  S,  the  program  gives  estimates  of  the  model 
parameters  ALPHA,  the  model  parameter  BETA,  the  total  number  of  faults,  the  number  of  remaining 
faults,  and  the  three  previously  defined  values.  For  the  meaning  of  the  model  parameters,  refer  to 
Reference  2. 

The  same  prediction  statistics  as  considered  by  other  models  (refer  to  Paragraphs  11.3.1  and 
11.3.2)  are  generated  by  Schneidewind’s  model.  Therefore,  the  same  comments  made  in  those 
paragraphs  are  applicable  here  for  both  the  requested  inputs  and  associated  outputs. 


11.3.5  S-Shaoed  Reliability  Growth  Model 

The  program  gives  estimates  of  the  proportionality  constant,  the  total  number  of  faults,  and  the 
number  of  remaining  faults.  The  associated  95-percent  confidence  intervals  are  also  given.  For  a 
description  of  the  model,  see  Reference  6.  This  model  is  a  good  candidate  for  fitting  a  data  set  if  a 
"learning  curve"  scenario  applies  (i.e.,  the  number  of  faults  found  slowly  ramps  up  as  the  testers  become 
familiar  with  the  code,  accelerates  quickly  up,  and  then  levels  off  as  the  remaining  faults  become  more 
difficult  to  find). 

As  in  previous  models,  the  user  can  obtain  the  expected  number  of  errors  in  the  next  testing  period 
and/or  the  number  of  periods  to  uncover  the  next  M  faults.  Considerations  concerning  the  length  are  the 
same  as  given  previously  (see  Paragraphs  11.3.1  and  11.3.2  for  example).  A  third  reliability  indicator 
is  also  provided.  It  is  the  software  reliability  for  the  next  testing  length.  After  entering  the  length,  the 
user  is  shown  an  estimate  of  the  probability  that  the  software  will  operate  without  failure  for  that  time 
frame.  An  example  would  be  if  the  user  had  entered  the  time  intervals  of  the  original  data  set  to 
represent  a  month  and  the  user  entered  a  six  for  the  prompted  length,  the  program  would  give  an  estimate 
that  the  software  would  operate  without  failure  for  a  period  of  six  months.  These  statistics  can  be 
extremely  useful  in  doing  tradeoff  analyses  to  determine  optimal  release  time  or  determining  when  an 
existing  operational  program  should  be  rewritten. 
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CHAPTER  12 

ANALYSES  OF  MODEL  FIT  SUB-MODULE 


12.1  INTRODUCTION 

The  Analyses  of  Model  Fit  sub-module  of  the  Model  Execution  modules  (Chapters  10  and  11) 
provides  four  methods  for  determining  how  well  a  model  fits  the  data.  This  entire  area  of  validation  of 
a  model  fit  is  of  the  utmost  importance;  predictions  and  estimations  of  a  particular  model  should  never 
be  taken  as  reflective  of  the  data  until  some  (all)  of  these  analyses  have  bron  conducted. 

For  interval  data,  the  chi-square  Goodness-of-Fit  statistic  can  be  obtained.  The  computed  values 
(in  conjunction  with  a  chi-square  table)  provide  a  firm  mathematically  and  statistically  based  analysis  of 
the  data.  For  execution  time  data,  the  Kolmogorov  Distance  is  calculated  and  die  distance  is  internally 
evaluated  to  determine  if  there  is  a  significant  amount  of  randomness  in  the  residuals  to  indicate  that  the 
model  may  fit  the  data.  (Refer  to  Section  12.3  for  more  details  on  die  Kolmogorov  Distance.)  The 
remaining  three  analyses  are  more  subjective,  calling  on  die  user  to  determine  if  patterns  exist  in  the 
predictions.  These  include  data  listings  of  the  original,  predicted,  and  residuals  data,  plots  of  die  original 
and  predicted  data,  and  plots  of  die  residuals.  One  is  looking  for  both  good  tracking  of  die  observed 
values  by  the  model  and  no  apparent  "trend"  in  the  plot  of  the  residuals.  If  the  model  is  adequately 
capturing  most  of  the  variation  in  die  data,  what  is  left  should  be  primarily  "noise".  Thus,  no  pattern 
should  be  present. 

The  plots  are  the  result  of  an  internal  line  printer  plotter.  The  internal  plotter  produces  very  crude 
graphs  to  assist  the  user  in  quick  interactive  examinations  of  the  data  and  model  fit.  It  is,  however, 
highly  recommended  that  users  make  use  of  the  optional  Plot  file  (Chapter  2,  step  d).  Examples  of  this 
file  can  be  found  in  Appendixes  C  and  D. 


12.2  EXECUTION  FLOW 

This  section  explains  the  prompts  and  inputs  for  the  analyses  of  die  model  fit  portion  of  SMERFS. 
The  placement  of  and  the  print  in  the  figures  conform  to  the  strategy  introduced  in  Sections  1.5  and  1.6. 

a.  Analyses  Option  Selection 

The  program  prompts  the  user  for  the  type  of  analysis  to  be  performed  (Figure  12-1).  If  an 
invalid  option  number  is  entered,  the  same  list  and  second  prompt  are  output. 
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UTU  ANALYSES  OPTION,  OR  KERO  FOR  A  LIST. 

THE  AVAILABLE  ANALYSES  OPTIONS  ARE: 

1  GOODNESS-OF-FIT  TESTS 

2  ORIGINAL,  PREDICTED,  AND  RESIDUAL  DATA  LISTINGS 

3  PLOT  OF  ORIGINAL  AND  PREDICTED  DATA 

4  PLOT  OF  THS  RESIDUAL  DATA 

5  RETURN  TO  THE  MODEL  EXECUTION  MENU 
ENTER  ANALYSES  OPTION. 


FIGURE  12-1.  ANALYSES  OF  MODEL  FIT  MENU 


b.  Goodness -of-Fit  Test  for  Execution  Time  Data  (Kolmogorov  Distance) 

If  the  entered  analysis  option  number  is  one  and  TBF  data  were  analyzed  (data  type  was 
entered  as  a  one,  two,  or  three),  SMERFS  issues  one  of  two  possible  messages  (Figure  12-2) 
reflecting  if  the  model  may  provide  an  adequate  fit.  (Refer  to  Section  12.3  for  more  details 
on  the  Kolmogorov  Distance  and  its  utility.)  Upon  completion,  control  returns  to  step  a. 


It  thm  computed  distance  is  significant,  thmnt 

THE  KOLMOGOROV  DISTANCE  IS  aaaaaaaaama aaa,  BASED  UPON  ill  DATA 
POINTS.  THAT  DISTANCE  IS  SIGNIFICANT  AT  THE  0.05  LEVEL)  THERE¬ 
FORE,  IT  APPEARS  THE  MODEL  MAY  NOT  PROVIDE  AN  ADEQUATE  FIT. 

Kiss,  if  the  modal  may  provide  an  adequate  fit,  than: 

THE  KOLMOGOROV  DISTANCE  IS  MMMtMtMM,  BASED  UPON  ill  DATA 
POINTS.  THAT  DISTANCE  IS  NOT  SIGNIFICANT  AT  THE  0.05  LEVEL; 
THEREFORE,  IT  APPEARS  THE  MODEL  MAY  PROVIDE  AN  ADEQUATE  FIT. 

End  it 


FIGURE  12-2.  KOLMOGOROV  DISTANCE  OUTPUT 


c.  Goodness-of-Fit  Test  for  Interval  Data  (Chi-Square) 

If  the  entered  analysis  option  number  is  one  and  interval  data  were  analyzed  (data  type  was 
entered  as  a  four),  SMERFS  prompts  for  the  cell  combination  frequency  number  to  be  used 
in  constructing  the  chi-square  interval  partitions  (Figure  12-3).  If  die  last  chi-square  interval 
could  not  be  made  to  contain  the  user-specified  ceil  frequency,  die  warning  lines  from  die 
figure  are  output,  containing  the  value  of  the  last  chi-square  interval.  The  actual  chi-square 
statistic  and  die  computed  degrees -of-freedom  are  then  output,  and  the  user  is  allowed  to 
repeat  the  execution  with  a  different  cell  combination  frequency.  (Refer  to  Section  12.3  for 
a  description  of  these  values.)  Upon  completion,  control  returns  to  step  a. 
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ERROR:  If  the  calculated  degrees-of-ffeedom  was  computed  to  be  less  than  one,  the 

message  shown  at  the  bottom  of  the  figure  is  issued. 


BITER  THE  CELL  COMBINATION  FREQUENCY  (THE  STANDARD  IS  FIVE) ,  OR 
A  MINUS  ONE  TO  INDICATE  NO  CELL  COMBINATIONS. 

If  the  cell  combination  could  not  be  achieved,  then: 

THE  CELL  COMBINATION  HAS  NOT  ACHIEVED  FOR  THE  LAST  INTERVAL; 
THE  GREATEST  POSSIBLE  VALUE  HAS  eeeeeeeeeeeeee . 

End  if 

THE  CHI-SQUARE  STATISTIC  IS  eeeeeeeeeeeeee  HITH  iiiii  DEGREES- 
OF- FREEDOM . 

**THE  CHI-SQUARE  STATISTIC  CANNOT  BE  CALCULATED;  THE  DEGREES-OF- 
FREEDOM  HAS  CALCULATED  TO  BE  LESS  THAN  ONE. 

ENTER  ONE  TO  TRY  ANOTHER  COMBINATION  FREQUENCY;  ELSE  ZERO. 


FIGURE  12-3.  CHI-SQUARE  PROMPTS  AND  OUTPUT 


d.  Original,  Predicted,  and  Residual  Data  Listing 

If  the  entered  analysis  option  number  is  two,  the  listing  (containing  the  original,  predicted, 
and  residual  data)  is  output  (Figure  12-4)  and  control  returns  to  step  a. 


NUMBER 

ORIGINAL  DATA 

PREDICTED  DATA 

RESIDUAL  DATA 

ill 

eeeeeeeeeeeeee 

• 

eeeeeeeeeeeeee 

• 

eeeeeeeeeeeeee 

• 

iii 

* 

eeeeeeeeeeeeee 

e 

eeeeeeeeeeeeee 

♦ 

eeeeeeeeeeeeee 

FIGURE  12-4.  LIST  ORIGINAL,  PREDICTED,  AND  RESIDUAL  VALUES 


e.  Plot  of  Original  and  Predicted  Data 

If  the  entered  analysis  option  number  is  three,  then  SMERFS  prompts  for  the  plot  title 
(Figure  12-5).  For  correct  centering  on  the  plot,  the  title  should  be  centered  in  the 
30-character  field. 

The  plot  is  then  generated  (Figure  12-5),  including  the  user-specified  title  and  the  program- 
determined  axis  labels.  The  units  for  the  TBF  measurement  will  also  be  placed  on  the  plot 
as  a  second  y-axis  label.  This  second  field  will  be  left  blank  for  interval  data. 
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The  plotter  can  only  reflect  a  maximum  of  SO  elements  on  the  x-axis.  The  message  shown 
at  the  bottom  of  the  figure  not  only  provides  for  a  pause  between  plots,  but  also  allows  for 
termination  of  the  plotting.  After  the  plotting,  control  returns  to  step  a. 

NOTE:  The  minimum  and  maximum  values  for  the  y-axis  are  defined  over  all  values 

to  be  plotted,  not  just  those  reflected  in  the  plot  of  SO  points. 


ENTER  A  PLOT  TITLE  (UP  TO  30  CHARACTERS) . 


auaaa 

aaaaaaa 


— emm mmaoomem 


*  ■  original  data 
P  •  predicted  value 

$  ■  *  and  P  too  close  to  distinguiah  on  plot 


+ - + - ♦ - ♦ - ^ - - + 

iii  iii  iii  iii  iii  iii 


ENTER  ONE  TO  VIEW  THE  NEXT  SECTION;  ELSE  ZERO  TO  EXIT  PLOTTER. 


FIGURE  12-5.  ORIGINAL  AND  PREDICTED  PLOT  FORMAT  AND  PROMPTS 


f.  Plot  of  the  Residual  Data 

If  the  entered  analysis  option  number  is  four,  then  SMERFS  prompts  for  the  plot  title 
(Figure  12-6).  For  correct  centering  on  the  plot,  the  title  should  be  centered  in  the 
30-character  field.  Additionally,  if  the  sample  size  is  greater  than  six,  SMERFS  prompts 
the  user  to  determine  if  smoothing  of  the  residua)  data  is  desired  with  the  second  prompt 
shown  in  Figure  12-6.  The  smoothing  technique  is  described  in  Reference  7. 

The  plot  is  then  generated  (Figure  12-6),  including  the  user-specified  title  and  the  program- 
determined  axis  labels.  The  units  for  the  TBF  measurement  will  also  be  placed  on  the  plot 
as  a  second  y-axis  label.  This  second  field  will  be  left  blank  for  interval  data. 
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The  plotter  can  only  reflect  a  maximum  of  SO  elements  on  the  x-axis.  The  message  shown 
at  the  bottom  of  the  figure  not  only  provides  for  a  pause  between  plots,  but  also  allows  for 
termination  of  the  plotting.  After  the  plotting,  control  returns  to  step  a. 

NOTE:  The  minimum  and  maximum  values  for  the  y-axis  are  defined  over  all  values 

to  be  plotted,  not  just  those  reflected  in  the  plot  of  SO  points. 


ENTER  A  PLOT  TITLE  (UP  TO  30  CHARACTERS) . 
ENTER  ONE  TO  SMOOTH  THE  RESIDUALS;  ELSE  ZERO. 


— — — — a— eee 


* 


difference  between  the  original  end  the 
predicted  values  from  the  model 


aaaaaaa 


eeeeeeeeeeeeee  I 

+ - ♦ — - + - +~ — — - + — - ♦ 

iii  iii  iii  iii  iii  iii 

aaaaaaaaaa 

ENTER  ONE  TO  VIEW  THE  NEXT  SECTION;  ELSE  ZERO  TO  EXIT  PLOTTER. 


FIGURE  12-6.  RESIDUAL  PLOT  FORMAT  AND  PROMPTS 


g.  Return  to  the  Main  Module  (and  Placing  Data  on  the  Plot  File) 

If  the  entered  analysis  option  number  is  five,  then  the  analyses  of  model  fit  is  over  and  the 
user  has  requested  a  return  to  the  Model  Menu  (Chapter  10,  step  b  for  execution  time  data 
or  Chapter  11,  step  a  for  interval  data).  Before  the  return  is  performed,  SMERFS  checks 
to  see  if  the  optional  Plot  file  was  established  (Chapter  2,  step  d).  If  established,  SMERFS 
allows  the  user  the  opportunity  to  place  the  Goodness-of-Fit  statistic  and  the  pertinent  data 
on  the  file  (Figure  12-7).  If  the  response  indicates  a  desire  to  place  these  data  on  the  file, 
a  40-character  title  is  then  requested.  Then  the  title,  axis  labels,  and  all  data  are  written  to 
the  file.  (Refer  to  Section  13.3  for  content  description  and  Appendixes  C  and  D  for 
examples.)  Upon  completion,  control  returns  to  the  Model  Menu  (Chapter  10,  step  b  for 
execution  time  data  or  Chapter  11,  step  a  for  interval  data). 


NSWCDD  TR  84-373 


If  tha  optional  SMERJFS  plot  film  van  mmtabliahmd ,  than : 

BITER  (MB  TO  PLACE  THE  PLOT  AND  GOODNESS-OT-FXT  STATISTIC  Ml  TEE 
OPTIONAL  SHEETS  PLOT  PILE;  ELSE  ZERO. 

If  a  poaitivo  raaponaa  was  input ,  thorn 

PLEASE  ENTER  A  PLOT  TITLE  TOR  ALL  DATA  (UP  TO  40  CBARACTERS) . 
End  if 
End  if 


FIGURE  12-7.  PLACING  THE  DATA  ON  THE  SMERFS  PLOT  FILE 
12.3  VALUES  DESCRIPTIONS 

The  Kolmogorov  distance  is  a  measure  of  the  discrepancy  between  the  sample  cumulative 
distribution  function  and  the  model’s.  It  is  the  maximum  "gap”  between  the  sample’s  cumulative 
distribution  function  as  represented  by  a  step  function  and  the  theoretical  one  based  upon  the  model.  If 
this  difference  is  too  large,  the  model  may  be  inadequate.  Note,  the  Kolmogorov  distance  is  not  a  true 
K-S  distance  test  as  model  parameters  have  been  replaced  by  their  estimates.  The  distributional 
assumptions  of  the  K-S  distance  may  therefore  not  be  satisfied.  The  user  needs  to  keep  in  mind  that  only 
an  approximate  test  is  being  performed.  Past  studies  have  shown,  for  various  distributions,  that  the  actual 
significance  level  of  the  test  may  be  smaller  than  what  is  used.  This  tends  to  lead  to  conservative  tests 
(i.e.,  rejecting  the  model  adequacy  when  the  model  is  appropriate  significantly  fewer  times  than  normal). 

For  the  implementation  of  the  chi-square  Goodness-of-Fit  test  within  the  program,  SMERFS  allows 
the  user  to  input  a  value  such  that  each  of  the  expected  cell  frequencies  will  be  greater  than  or  equal  to 
that  value  (Figure  12-3).  Suggested  values  are  -1.0  (each  data  point  is  treated  separately,  i.e.,  no 
combinations  are  formed)  and  3.0  (the  usual  value  seen  in  the  literature  on  the  chi-square  test).  The 
chi-square  test  (Reference  8)  rejects  the  null  hypothesis  that  the  data  follow  the  fitted  model  if  the  value 
of  the  chi-square  statistic  is  larger  than  the  upper  tail  X*(l  -  a,df)  of  a  chi-square  distribution  with 
degrees-of-freedom  of  df  and  a  significance  level  of  a.  For  example,  if  a  =  0.0S  (it  is  acceptable  to  take 
a  chance  of  0.05  of  rejecting  the  null  hypothesis  when  it  is  true),  df  =  10,  and  the  output  chi-square 
value  is  14.62;  then  X^l  -  a,df)  =  X*(0.95,10)  =  18.3,  so  the  null  hypothesis  would  be  accepted.  The 
data  do  tend  to  follow  the  model. 
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CHAPTER  13 

SMERFS  TERMINATION  AND  FOLLOWUP 


13.1  INTRODUCTION 

If  die  entered  Main  module  option  number  is  nine,  SMERFS  responds  with  a  message  of  program 
termination  (Figure  13-1)  and  returns  the  user  to  the  command  mode  of  die  terminal. 


FIGURE  13-1.  SMERFS  TERMINATION 


13.2  COMMANDS 

The  followup  commands  only  pertain  to  executions  in  which  output  files  were  generated.  These 
include  the  data  file(s),  the  History  file,  and  the  Plot  file.  If  these  files  were  not  generated,  this  section 
may  be  ignored. 

If  output  files  were  generated,  probably  some  additional  instructions  from  the  operating  system 
level  of  die  target  computer  are  required.  Although  the  specific  commands  vary  from 
computer-to-computer,  the  basic  actions  are  placing  the  developed  files  on  a  permanent  storage  media  and 
removing  any  redundant  files  from  that  area.  The  first  key  question  to  answer  prior  to  session 
termination  is  "Are  the  data.  History,  and  Plot  files  in  an  area  that  will  not  be  lost  when  the  session  is 
terminated?”  For  die  CDC  170/875  (which  has  local  and  permanent  files),  this  implies  that  die  local  files 
(generated  by  the  program)  must  be  "saved”  as  permanent  files.  For  the  VAX  and  many  IBM-compatible 
PCs,  no  additional  action  is  needed. 

Conversely,  if  the  program  was  terminated  because  of  errors;  and  the  data.  History,  and  Plot  files 
are  not  desired,  the  computers  with  local  and  permanent  files  require  no  additional  action.  The  applicable 
files  must  be  "deleted"  from  the  computers  which  have  only  permanent  files.  It  is  also  emphasized  that 
die  History  and  Plot  files  are  opened  as  FORTRAN  Scratch  files  when  they  are  not  requested  for 
retention.  This  causes  the  creation  of  two  files  (usually  started  with  "FOR"  or  "ZZ”  and  followed  by 
a  series  of  characters)  when  an  abnormal  shutdown  of  the  program  occurs;  they  should  be  deleted,  also. 

A  second  concern  should  be  ensuring  that  an  adequate  number  of  backup  copies  is  available, 
without  burdening  the  system  with  files  that  are  no  longer  needed.  The  user  must  determine  at  what  point 


13-1 


NSWCDD  TR  84-373 


die  files  are  do  longer  to  be  kept.  Obviously,  if  the  data  file(s)  is  updated  daily  and  the  projected  testing 
span  is  over  a  year,  it  would  waste  the  computer  storage  area  if  all  files  were  retained.  Some  previous 
files  should  be  retained  in  order  to  have  the  ability  to  recreate  the  data  base  from  a  certain  point.  That 
point  must  be  determined  by  the  user  on  a  per  analysis  basis. 


13.3  FORMAT  OF  THE  OPTIONAL  PLOT  FILE 

The  optional  SMERFS  Plot  file  is  designed  to  be  easily  input  to  a  separate  graphics  program  to 
produce  high-quality  plots  of  die  data.  Each  selected  set  of  data  contains  a  fixed  amount  of  information. 
For  easier  comprehension  of  this  description,  refer  to  Appendixes  C  and  D  for  CPU  TBF  and  interval 
data  analysis  examples,  respectively. 

The  first  line  contains  the  40-character  thle  for  the  data  (as  entered  in  Chapter  12,  step  g),  the 
number  of  data  points,  and  the  units  of  measurement.  The  third  field  is  left  blank  for  interval  data.  The 
format  of  this  line  is: 

(1X,A40,5X,I5,3X,A7) 

The  next  line  contains  the  labels  for  die  axes.  The  "FAILURE  NUMBER"  and  "WC 
EXPENDITURE"  labels  are  used  for  WC  TBF  data  analysis.  The  "CPU  EXPENDITURE"  label 
replaces  die  Y-axis  label  when  CPU  TBF  data  analysis  is  done.  The  "INTERVAL  NUMBER",  "FAULT 
COUNT",  and  "TESTING  LENGTH"  labels  are  used  for  interval  data  analysis.  The  format  of  this  line 
is: 

(1X,A15,5X,A  15.5X,  A 15) 

The  next  two  lines  contain  die  results  from  a  Goodness -of-Fit  analysis.  For  die  TBF  data  analysis, 
the  lines  are  set  to  contain  the  Kolmogorov  distance  and  die  determined  adequacy  of  the  model  fit.  The 
formats  of  these  lines  are  either: 

C  THE  CALCULATED  KOLMOGOROV  DISTANCE  IS\E15.8) 

C  THE  VALUE  MAY  PROVIDE  AN  ADEQUATE  FIT  AT  THE  0.05  LEVEL’) 

or 

(’  THE  CALCULATED  KOLMOGOROV  DISTANCE  IS\E15.8) 

(’  THE  VALUE  MAY  NOT  PROVIDE  AN  ADEQUATE  FIT  AT  THE  0.05  LEVEL’) 

depending  on  the  significance  of  the  distance  at  the  0.05  level. 

For  the  interval  data  analysis,  die  lines  are  set  to  an  error  message  (followed  by  a  blank  line)  if 
die  degrees-of-freedom  is  computed  to  be  less  than  one  (using  a  cell  combination  frequency  of  five).  The 
format  of  the  error  message  line  is: 

C  CHI-SQUARE  STATISTIC  NOT  AVAILABLE’) 
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If  the  error  was  not  encountered,  then  the  chi-square  statistic  and  the  degrees -of-freedom  are  output.  The 
format  of  this  line  is: 

(’  CHI-SQUARE  STATISTIC  IS\E15.8,’  WITH, 16,*  DEGREES-OF-FREEDOM’) 

The  next  line  (for  the  TBF  data  analysis)  is  set  to  indicate  that  the  cell  combination  frequency  (S) 
could  not  be  achieved  for  the  last  cell;  this  line  shows  the  largest  achieved  value  for  the  last  cell.  The 
format  of  this  line  is: 

(’  THE  GREATEST  VALUE  FOR  THE  LAST  CELL  WAS\E15.8) 

If  die  cell  combination  frequency  was  achieved,  the  line  is  left  blank. 

The  remaining  lines  for  this  set  of  data  contain  the  original  data,  the  smoothed  original  data,  the 
predicted  data,  the  residual  data,  and  the  smoothed  residual  data.  The  five  "blocks"  of  data  are  prefixed 
with  an  appropriate  tide  line  from  the  formats: 

(’  VALUES  FOR  RAW  DATA  PLOT’) 

C  VALUES  FOR  SMOOTHED  RAW  DATA  PLOT’) 

(’  VALUES  FOR  PREDICTED  DATA  PLOT’) 

C  VALUES  FOR  RESIDUAL  DATA  PLOT’) 

C  VALUES  FOR  SMOOTHED  RESIDUAL  DATA  PLOT) 

After  die  first  tide,  either  die  raw  TBF  data  or  the  interval  fault  counts  and  testing  lengths  are  listed 
(one  failure  or  interval  per  line).  The  format  of  the  lines  is  (1X,E15.8)  or  (1X,2E15.8),  respectively. 
The  other  four  sets  of  data  are  all  listed  (one  per  line)  with  a  format  of  (1X,E15.8). 

The  only  deviation  from  die  prior  description  of  the  five  blocks  occurs  when  the  sample  size  is  less 
than  or  equal  to  six.  Under  that  condition,  the  second  and  fifth  tides  are  replaced  with: 

C  ***  SAMPLE  SIZE  TOO  SMALL  TO  SMOOTH  ***’) 
and  Dfl  data  values  are  listed  under  those  tides. 

The  next  set  of  data  (if  available)  begins  with  a  new  40-character  dde  and  sample  size. 
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Sunpl*  CPU  TBP  Plot  Fils  17  MINUTES 

FAILURE  NUMBER  CPU  EXPENDITURE 

THE  CALCULATED  KOLMOGOROV  DISTANCE  IS  .40834229E+00 
THE  MODEL  MAY  NOT  PROVIDE  AN  ADEQUATE  FIT  AT  THE  0.05  LEVEL 
VALUES  FOR  RAN  DATA  PLOT 
. 10000000B+0 1 
. 40000000B+02 


. 60000000E+03 
. 90000000E+03 

VALUES  FOR  SMOOTHED  RAW  DATA  PLOT 
-.27500000B+01 
. 14000000E+02 


.75000000E+03 
. 90000000E+03 

VALUES  FOR  PREDICTED  DATA  PLOT 
. 15094311E+02 
. 19899128E+02 


. 95306927E+03 
•12564500E+04 

VALUES  FOR  RESIDUAL  DATA  PLOT 
-.14094311E+02 
. 20100872E+02 


35306927E+03 
3564500 1B+03 

VALUES  FOR  SMOOTHED  RESIDUAL  DATA  PLOT 
-.13243311E+02 
-.81920603E+01 


-.27774257E+03 

-.35645001E+03 
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SAMPLE  INTERVAL  DATA  ANALYSIS 
PLOT  FILE 
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Sample  Interval  Plot  File  30 

INTERVAL  NUMBER  FAULT  COUNT  TESTING  LENGTH 

CHI-SQUARE  STATISTIC  IS  . 39882 601E+02  WITH  21  DEGREES-OF-FREEDOM 

VALUES  FOR  RAN  DATA  PLOT 

. 20000000E+02  . 10000000E+0 1 

. 18000000E+02  . 10000000E+01 


. 00000000E+00  . 10000000E+01 

. 10000000E+01  . 10000000E+01 

VALUES  FOR  SMOOTHED  RAH  DATA  PLOT 
. 22000000E+02 
. 22000000E+02 


. 00000000E+00 
. 00000000E+00 

VALUES  FOR  PREDICTED  DATA  PLOT 
. 29821787E+02 
. 27064728E+02 


. 19719396E+01 
. 17896314E+01 

VALUES  FOR  RESIDUAL  DATA  PLOT 
-.98217869E+01 
-.906472 76E+01 


-. 19719396E+01 
-.78963144E+00 

VALUES  FOR  SMOOTHED  RESIDUAL  DATA  PLOT 
-.81962454E+01 
6002909 5E+01 


- . 13823581E+01 
89518715E+00 
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